[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:41 dist-test-slave-vj6n krb5kdc[13352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:41 dist-test-slave-vj6n krb5kdc[13352](info): set up 2 sockets
Sep 24 16:06:41 dist-test-slave-vj6n krb5kdc[13352](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 16:06:42.439775 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:42 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:42.561780 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46113
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:46113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:42.699622 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:42.699968 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:42.700052 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:42.704226 13365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:42.704317 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:42.704356 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:42.704422 13365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:42.704463 13365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:42.710078 13365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:46113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:42.711340 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:42.712513 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:42.723179 13371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:42.723371 13365 server_base.cc:1034] running on GCE node
W20260924 16:06:42.723248 13370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:42.723232 13373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:42.724093 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:42.725356 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:42.726567 13365 hybrid_clock.cc:648] HybridClock initialized: now 1790266002726520 us; error 54 us; skew 500 ppm
Sep 24 16:06:42 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266002, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:42.729598 13365 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:42.730715 13365 webserver.cc:460] Webserver started at http://127.13.8.126:41251/ using document root <none> and password file <none>
I20260924 16:06:42.731292 13365 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:42.731355 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:42.731577 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:42.733803 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "5c3211e71660423082ff50688ab657bc"
format_stamp: "Formatted at 2026-09-24 16:06:42 on dist-test-slave-vj6n"
server_key: "2c0f3d807b7356b969dee763723c278e"
server_key_iv: "34a92e56fb99b8b34a1a0e60069e2aaf"
server_key_version: "encryptionkey@0"
I20260924 16:06:42.734498 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "5c3211e71660423082ff50688ab657bc"
format_stamp: "Formatted at 2026-09-24 16:06:42 on dist-test-slave-vj6n"
server_key: "2c0f3d807b7356b969dee763723c278e"
server_key_iv: "34a92e56fb99b8b34a1a0e60069e2aaf"
server_key_version: "encryptionkey@0"
I20260924 16:06:42.738643 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260924 16:06:42.741266 13380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:42.742686 13365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:06:42.742873 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "5c3211e71660423082ff50688ab657bc"
format_stamp: "Formatted at 2026-09-24 16:06:42 on dist-test-slave-vj6n"
server_key: "2c0f3d807b7356b969dee763723c278e"
server_key_iv: "34a92e56fb99b8b34a1a0e60069e2aaf"
server_key_version: "encryptionkey@0"
I20260924 16:06:42.742987 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:42.764091 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:42.767669 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:42.767939 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:42.781643 13365 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:46113
I20260924 16:06:42.781653 13432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:46113 every 8 connection(s)
I20260924 16:06:42.782786 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:42.787135 13433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:42.790438 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13365
I20260924 16:06:42.790618 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:42.792349 13345 external_mini_cluster.cc:1428] Setting key 062517aa51597c9343f4cd4958160da4
I20260924 16:06:42.794430 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc: Bootstrap starting.
I20260924 16:06:42.797161 13433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:42.798014 13433 log.cc:824] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:42.800139 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc: No bootstrap required, opened a new log
I20260924 16:06:42.808663 13433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } }
I20260924 16:06:42.809001 13433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:42.809101 13433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3211e71660423082ff50688ab657bc, State: Initialized, Role: FOLLOWER
I20260924 16:06:42.809572 13433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } }
I20260924 16:06:42.809697 13433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:42.809773 13433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:42.809898 13433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:42.811022 13433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } }
I20260924 16:06:42.811398 13433 leader_election.cc:302] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c3211e71660423082ff50688ab657bc; no voters:
I20260924 16:06:42.811755 13433 leader_election.cc:288] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:42.811822 13438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:42.812038 13438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [term 1 LEADER]: Becoming Leader. State: Replica: 5c3211e71660423082ff50688ab657bc, State: Running, Role: LEADER
I20260924 16:06:42.812366 13438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } }
Sep 24 16:06:42 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:42.813114 13433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:42.814131 13439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c3211e71660423082ff50688ab657bc. Latest consensus state: current_term: 1 leader_uuid: "5c3211e71660423082ff50688ab657bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } } }
I20260924 16:06:42.814317 13439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:42.814746 13442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:42.815351 13440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c3211e71660423082ff50688ab657bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3211e71660423082ff50688ab657bc" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46113 } } }
I20260924 16:06:42.816460 13440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:42.818785 13442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:42.820991 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:42.796596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.13.8.126:46113)
0924 16:06:42.797162 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:42.797183 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:42.798453 (+ 1270us) server_negotiation.cc:407] Connection header received
0924 16:06:42.802591 (+ 4138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:42.802615 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:42.803012 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:42.803403 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:42.805529 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:42.806743 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:42.807996 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:42.808677 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:42.808845 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:42.813505 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:42.813515 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:42.813533 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:42.813568 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:42.815722 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:42.816353 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:42.816362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:42.816369 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:42.816522 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:42.816939 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:42.816943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:42.816945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:42.817433 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:42.817625 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:42.819726 (+ 2101us) server_negotiation.cc:299] Negotiation successful
0924 16:06:42.820008 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":171,"threads_started":1}
I20260924 16:06:42.825134 13442 catalog_manager.cc:1347] Generated new cluster ID: f2c0a6dca0494b7b8b9fa28f53b0bf48
I20260924 16:06:42.825193 13442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:42.851529 13442 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:42.852593 13442 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:42.859882 13442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c3211e71660423082ff50688ab657bc: Generated new TSK 0
I20260924 16:06:42.860684 13442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:06:42.897215 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:43.011690 13460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:43.011982 13460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:43.012068 13460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:43.016261 13460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:43.016326 13460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:43.016384 13460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:06:43.021633 13460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:43.022676 13460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:43.023628 13460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:43.033242 13466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.033242 13465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.033340 13468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:43.033406 13460 server_base.cc:1034] running on GCE node
I20260924 16:06:43.034035 13460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:43.034686 13460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:43.035874 13460 hybrid_clock.cc:648] HybridClock initialized: now 1790266003035844 us; error 49 us; skew 500 ppm
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:43.038523 13460 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:06:43.039402 13460 webserver.cc:460] Webserver started at http://127.13.8.65:36441/ using document root <none> and password file <none>
I20260924 16:06:43.039963 13460 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:43.040050 13460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:43.040295 13460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:43.042450 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "6f66b8c1bae34ff78d96d8d3bb61c3ac"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "de7a2f3db9a3f1b3b32f65d7fde31b46"
server_key_iv: "2c87685fd5858953e772b1e2c4ddd1d1"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.043090 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "6f66b8c1bae34ff78d96d8d3bb61c3ac"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "de7a2f3db9a3f1b3b32f65d7fde31b46"
server_key_iv: "2c87685fd5858953e772b1e2c4ddd1d1"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.046839 13460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:06:43.049417 13475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.050686 13460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:43.050815 13460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "6f66b8c1bae34ff78d96d8d3bb61c3ac"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "de7a2f3db9a3f1b3b32f65d7fde31b46"
server_key_iv: "2c87685fd5858953e772b1e2c4ddd1d1"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.050926 13460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:43.063699 13460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:43.066560 13460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:43.066766 13460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:43.067350 13460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:43.068255 13460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:43.068328 13460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.068399 13460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:43.068441 13460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.078896 13460 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:44299
I20260924 16:06:43.078946 13589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:44299 every 8 connection(s)
I20260924 16:06:43.079767 13460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:06:43.083199 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13460
I20260924 16:06:43.083314 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:06:43.083585 13345 external_mini_cluster.cc:1428] Setting key f45005179389db9999054ffdd7c9316c
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:06:43.093195 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.081508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:58799 (local address 127.13.8.126:46113)
0924 16:06:43.081686 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:43.081691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:43.082739 (+ 1048us) server_negotiation.cc:407] Connection header received
0924 16:06:43.083716 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:43.083720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:43.083775 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:43.083858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:43.085001 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.085523 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.086788 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.087031 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.087139 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.089490 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:43.089499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:43.089501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:43.089523 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:43.091079 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.091611 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.091615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.091617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.091697 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.092038 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.092042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.092044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.092220 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:43.092332 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:43.092845 (+ 513us) server_negotiation.cc:299] Negotiation successful
0924 16:06:43.093006 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:06:43.093971 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.081916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:46113 (local address 127.13.8.65:58799)
0924 16:06:43.082602 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:43.082641 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:43.083508 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:43.084028 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:43.084036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:43.084378 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:43.084852 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.084865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.085650 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.085655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.086654 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.086664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.087194 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.087198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.087381 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.087953 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:43.087969 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:43.089330 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:43.091207 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.091213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.091227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.091490 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.091791 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.091795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.091797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.091943 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.092333 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:43.092340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:43.092604 (+ 264us) client_negotiation.cc:769] Sending connection context
0924 16:06:43.092792 (+ 188us) client_negotiation.cc:240] Negotiation successful
0924 16:06:43.093144 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:06:43.095260 13590 heartbeater.cc:347] Connected to a master server at 127.13.8.126:46113
I20260924 16:06:43.095510 13590 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:43.096000 13590 heartbeater.cc:511] Master 127.13.8.126:46113 requested a full tablet report, sending...
I20260924 16:06:43.097539 13397 ts_manager.cc:194] Registered new tserver with Master: 6f66b8c1bae34ff78d96d8d3bb61c3ac (127.13.8.65:44299)
I20260924 16:06:43.098798 13397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:58799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:06:43.110397 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:43.223912 13595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:43.224211 13595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:43.224287 13595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:43.228292 13595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:43.228376 13595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:43.228466 13595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:06:43.233562 13595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:43.234629 13595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:43.235647 13595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:06:43.245819 13595 server_base.cc:1034] running on GCE node
W20260924 16:06:43.245837 13600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.245797 13601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.245859 13603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:43.246595 13595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:43.247259 13595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:43.248433 13595 hybrid_clock.cc:648] HybridClock initialized: now 1790266003248418 us; error 25 us; skew 500 ppm
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:43.251318 13595 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:06:43.252192 13595 webserver.cc:460] Webserver started at http://127.13.8.66:32869/ using document root <none> and password file <none>
I20260924 16:06:43.252776 13595 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:43.252866 13595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:43.253105 13595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:43.255309 13595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "bbd4c626ceb2471a9b7d51f2cf9150c9"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "2da328410d76fe45eba2c0352f2b9d87"
server_key_iv: "d37e7000c0a9f8605fdeb8ff1cbd8885"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.255875 13595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "bbd4c626ceb2471a9b7d51f2cf9150c9"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "2da328410d76fe45eba2c0352f2b9d87"
server_key_iv: "d37e7000c0a9f8605fdeb8ff1cbd8885"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.259660 13595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:06:43.262312 13610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.263572 13595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:06:43.263727 13595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "bbd4c626ceb2471a9b7d51f2cf9150c9"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "2da328410d76fe45eba2c0352f2b9d87"
server_key_iv: "d37e7000c0a9f8605fdeb8ff1cbd8885"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.263844 13595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:43.286778 13595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:43.290201 13595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:43.290437 13595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:43.291090 13595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:43.291955 13595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:43.292027 13595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.292097 13595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:43.292147 13595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.301596 13595 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:32925
I20260924 16:06:43.301611 13723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:32925 every 8 connection(s)
I20260924 16:06:43.302570 13595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:06:43.306514 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13595
I20260924 16:06:43.306697 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:06:43.306986 13345 external_mini_cluster.cc:1428] Setting key 0789026b275cd46fc188ea1f0501b7ad
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:06:43.317878 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.304401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:56637 (local address 127.13.8.126:46113)
0924 16:06:43.304549 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:43.304553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:43.305590 (+ 1037us) server_negotiation.cc:407] Connection header received
0924 16:06:43.306590 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:43.306593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:43.306649 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:43.306759 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:43.308215 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.309090 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.310526 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.310762 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.310851 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.313835 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:43.313842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:43.313847 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:43.313883 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:43.315631 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.316250 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.316256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.316259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.316363 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.316749 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.316754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.316757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.316991 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:43.317090 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:43.317653 (+ 563us) server_negotiation.cc:299] Negotiation successful
0924 16:06:43.317758 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:06:43.318897 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.304805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:46113 (local address 127.13.8.66:56637)
0924 16:06:43.305444 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:43.305486 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:43.306384 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:43.306940 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:43.306947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:43.307373 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:43.308039 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.308057 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.309242 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.309245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.310401 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.310415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.310919 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.310926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.311163 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.311819 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:43.311843 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:43.313591 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:43.315807 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.315817 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.315836 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.316138 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.316474 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.316476 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.316478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.316632 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.317095 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:43.317115 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:43.317410 (+ 295us) client_negotiation.cc:769] Sending connection context
0924 16:06:43.317581 (+ 171us) client_negotiation.cc:240] Negotiation successful
0924 16:06:43.317844 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:06:43.320259 13724 heartbeater.cc:347] Connected to a master server at 127.13.8.126:46113
I20260924 16:06:43.320515 13724 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:43.321050 13724 heartbeater.cc:511] Master 127.13.8.126:46113 requested a full tablet report, sending...
I20260924 16:06:43.322257 13397 ts_manager.cc:194] Registered new tserver with Master: bbd4c626ceb2471a9b7d51f2cf9150c9 (127.13.8.66:32925)
I20260924 16:06:43.322890 13397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:56637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:06:43.337627 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:43.450815 13729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:43.451093 13729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:43.451140 13729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:43.455566 13729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:43.455636 13729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:43.455699 13729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:06:43.461220 13729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:46113
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:43.462430 13729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:43.463508 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:43.473848 13735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.473877 13734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:43.473848 13737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:43.474136 13729 server_base.cc:1034] running on GCE node
I20260924 16:06:43.474593 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:43.475293 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:43.476491 13729 hybrid_clock.cc:648] HybridClock initialized: now 1790266003476456 us; error 65 us; skew 500 ppm
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:43.479453 13729 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:06:43.480330 13729 webserver.cc:460] Webserver started at http://127.13.8.67:43731/ using document root <none> and password file <none>
I20260924 16:06:43.480918 13729 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:43.481009 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:43.481243 13729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:43.483362 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "94de9781f880454b8515401ec9ce4e3b"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "f5bdccc440304a4324b03f3fd48d2205"
server_key_iv: "03bd011d13721da72e212b7f924db519"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.483938 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "94de9781f880454b8515401ec9ce4e3b"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "f5bdccc440304a4324b03f3fd48d2205"
server_key_iv: "03bd011d13721da72e212b7f924db519"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.487746 13729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:06:43.490139 13744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.491384 13729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:43.491516 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "94de9781f880454b8515401ec9ce4e3b"
format_stamp: "Formatted at 2026-09-24 16:06:43 on dist-test-slave-vj6n"
server_key: "f5bdccc440304a4324b03f3fd48d2205"
server_key_iv: "03bd011d13721da72e212b7f924db519"
server_key_version: "encryptionkey@0"
I20260924 16:06:43.491657 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:43.509584 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:43.512697 13729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:43.512904 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:43.513513 13729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:43.514405 13729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:43.514480 13729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.514557 13729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:43.514590 13729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:43.524291 13729 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:39451
I20260924 16:06:43.524310 13859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:39451 every 8 connection(s)
I20260924 16:06:43.525321 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:06:43.533957 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13729
I20260924 16:06:43.534051 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:06:43.534382 13345 external_mini_cluster.cc:1428] Setting key df97e6ee6a1a60690e9a1515fea7082f
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:43.538894 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.527252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37511 (local address 127.13.8.126:46113)
0924 16:06:43.527391 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:43.527394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:43.528378 (+ 984us) server_negotiation.cc:407] Connection header received
0924 16:06:43.529306 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:43.529310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:43.529365 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:43.529475 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:43.530745 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.531269 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.532434 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.532658 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.532792 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.535301 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:43.535305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:43.535308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:43.535332 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:43.536775 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.537290 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.537295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.537297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.537378 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.537722 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.537726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.537728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.537918 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:43.538028 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:43.538603 (+ 575us) server_negotiation.cc:299] Negotiation successful
0924 16:06:43.538720 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:06:43.539691 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.527557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:46113 (local address 127.13.8.67:37511)
0924 16:06:43.528220 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:43.528269 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:43.529078 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:43.529662 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:43.529670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:43.530028 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:43.530591 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.530605 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.531397 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.531400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.532304 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:43.532314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.532846 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.532851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:43.533015 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.533550 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:43.533566 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:43.535114 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:43.536896 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.536902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.536919 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.537177 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.537474 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:43.537478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:43.537480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:43.537626 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:43.538026 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:43.538033 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:43.538362 (+ 329us) client_negotiation.cc:769] Sending connection context
0924 16:06:43.538550 (+ 188us) client_negotiation.cc:240] Negotiation successful
0924 16:06:43.538871 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":113,"threads_started":1}
I20260924 16:06:43.540745 13861 heartbeater.cc:347] Connected to a master server at 127.13.8.126:46113
I20260924 16:06:43.540992 13861 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:43.541481 13861 heartbeater.cc:511] Master 127.13.8.126:46113 requested a full tablet report, sending...
I20260924 16:06:43.542529 13397 ts_manager.cc:194] Registered new tserver with Master: 94de9781f880454b8515401ec9ce4e3b (127.13.8.67:39451)
I20260924 16:06:43.543087 13397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:37511
I20260924 16:06:43.548658 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:43.575050 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.565293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54556 (local address 127.13.8.65:44299)
0924 16:06:43.565599 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:43.565614 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:43.565640 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:06:43.565730 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:43.565746 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:43.565952 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:43.566075 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:43.566794 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.567640 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.568502 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.568887 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.568973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.571588 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:43.571597 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:43.571613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:43.571636 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:43.573278 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.573677 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.573682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.573684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.573770 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.573996 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.573998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.573999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.574305 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:43.574472 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:43.574668 (+ 196us) server_negotiation.cc:299] Negotiation successful
0924 16:06:43.574842 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
W20260924 16:06:43.576118 13503 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54556
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:43.601980 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:43.593623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.13.8.65:44299)
0924 16:06:43.593764 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:43.593768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:43.593803 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:06:43.593947 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:43.593949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:43.594013 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:43.594087 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:43.594849 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.595430 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.596366 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:43.596585 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:43.596662 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:43.599033 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:43.599036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:43.599038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:43.599061 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:43.600450 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.600846 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.600848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.600849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.600918 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:43.601167 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:43.601169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:43.601170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:43.601313 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:43.601408 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:43.601665 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 16:06:43.601765 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 16:06:43.603117 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13460
I20260924 16:06:43.610877 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13595
I20260924 16:06:43.617100 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13729
I20260924 16:06:43.622529 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13365
2026-09-24T16:06:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2273 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13875](info): set up 2 sockets
Sep 24 16:06:43 dist-test-slave-vj6n krb5kdc[13875](info): commencing operation
krb5kdc: starting...
W20260924 16:06:44.670699 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:44 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:44.794523 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36657
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:36657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:44.903568 13888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:44.903810 13888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:44.903849 13888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:44.907693 13888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:44.907743 13888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:44.907763 13888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:44.907778 13888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:44.907792 13888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:44.912786 13888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:36657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:44.913769 13888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:44.914773 13888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:44.923411 13896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:44.923599 13888 server_base.cc:1034] running on GCE node
W20260924 16:06:44.923480 13893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:44.923437 13894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:44.924350 13888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:44.925460 13888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:44.926644 13888 hybrid_clock.cc:648] HybridClock initialized: now 1790266004926632 us; error 28 us; skew 500 ppm
Sep 24 16:06:44 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266004, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:44.929623 13888 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:44.930660 13888 webserver.cc:460] Webserver started at http://127.13.8.126:36687/ using document root <none> and password file <none>
I20260924 16:06:44.931267 13888 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:44.931360 13888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:44.931604 13888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:44.933753 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "779d6179412e4bc2972dbd5353ae3d89"
format_stamp: "Formatted at 2026-09-24 16:06:44 on dist-test-slave-vj6n"
server_key: "1dffc8502fa49d28ba3eb0e2e9bba557"
server_key_iv: "4ccba4a3ea9e26f40970372fde53100d"
server_key_version: "encryptionkey@0"
I20260924 16:06:44.934419 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "779d6179412e4bc2972dbd5353ae3d89"
format_stamp: "Formatted at 2026-09-24 16:06:44 on dist-test-slave-vj6n"
server_key: "1dffc8502fa49d28ba3eb0e2e9bba557"
server_key_iv: "4ccba4a3ea9e26f40970372fde53100d"
server_key_version: "encryptionkey@0"
I20260924 16:06:44.938407 13888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:06:44.940979 13903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:44.942483 13888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:06:44.942655 13888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "779d6179412e4bc2972dbd5353ae3d89"
format_stamp: "Formatted at 2026-09-24 16:06:44 on dist-test-slave-vj6n"
server_key: "1dffc8502fa49d28ba3eb0e2e9bba557"
server_key_iv: "4ccba4a3ea9e26f40970372fde53100d"
server_key_version: "encryptionkey@0"
I20260924 16:06:44.942777 13888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:44.963466 13888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:44.966684 13888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:44.966910 13888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:44.974180 13888 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:36657
I20260924 16:06:44.974205 13956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:36657 every 8 connection(s)
I20260924 16:06:44.975349 13888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:44.978315 13957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:44.980296 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13888
I20260924 16:06:44.980420 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:44.980685 13345 external_mini_cluster.cc:1428] Setting key 37d5e27a058eb70290149ac8c3918f7d
I20260924 16:06:44.985299 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89: Bootstrap starting.
I20260924 16:06:44.987711 13957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:44.988554 13957 log.cc:824] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:06:44 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:44.991127 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89: No bootstrap required, opened a new log
I20260924 16:06:44.995729 13957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } }
I20260924 16:06:44.995955 13957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:44.996004 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:44.982412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.13.8.126:36657)
0924 16:06:44.982902 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:44.982922 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:44.982959 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:06:44.983582 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:44.983597 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:44.983948 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:44.984511 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:44.985295 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:44.986448 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:44.987288 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:44.987813 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:44.987992 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:44.991107 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:44.991117 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:44.991132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:44.991160 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:44.992963 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:44.993550 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:44.993555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:44.993559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:44.993665 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:44.993971 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:44.993974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:44.993975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:44.994423 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:44.994580 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:44.994907 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 16:06:44.995219 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260924 16:06:44.996025 13957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779d6179412e4bc2972dbd5353ae3d89, State: Initialized, Role: FOLLOWER
I20260924 16:06:44.996609 13957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } }
I20260924 16:06:44.996776 13957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:44.996865 13957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:44.996990 13957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:44.998241 13957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } }
I20260924 16:06:44.998608 13957 leader_election.cc:302] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779d6179412e4bc2972dbd5353ae3d89; no voters:
I20260924 16:06:44.998981 13957 leader_election.cc:288] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:44.999084 13962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:44.999382 13962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [term 1 LEADER]: Becoming Leader. State: Replica: 779d6179412e4bc2972dbd5353ae3d89, State: Running, Role: LEADER
I20260924 16:06:44.999739 13962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } }
I20260924 16:06:45.000538 13957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:45.000929 13963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779d6179412e4bc2972dbd5353ae3d89. Latest consensus state: current_term: 1 leader_uuid: "779d6179412e4bc2972dbd5353ae3d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } } }
I20260924 16:06:45.001062 13963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:45.001479 13966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:45.001778 13964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779d6179412e4bc2972dbd5353ae3d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d6179412e4bc2972dbd5353ae3d89" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36657 } } }
I20260924 16:06:45.001868 13964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:45.005249 13966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:45.011193 13966 catalog_manager.cc:1347] Generated new cluster ID: 34e1f6662fcc4964989e1b18c85d5b86
I20260924 16:06:45.011284 13966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:45.029114 13966 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:45.030515 13966 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:45.059430 13966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779d6179412e4bc2972dbd5353ae3d89: Generated new TSK 0
I20260924 16:06:45.060387 13966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:06:45.090021 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:45.204118 13983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:45.204437 13983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:45.204530 13983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:45.208643 13983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:45.208724 13983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:45.208813 13983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:06:45.214637 13983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:45.215855 13983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:45.216964 13983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:45.226996 13991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.227013 13988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.227026 13989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:45.227421 13983 server_base.cc:1034] running on GCE node
I20260924 16:06:45.227792 13983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:45.228443 13983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:45.229674 13983 hybrid_clock.cc:648] HybridClock initialized: now 1790266005229635 us; error 48 us; skew 500 ppm
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:45.232426 13983 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:06:45.233268 13983 webserver.cc:460] Webserver started at http://127.13.8.65:37595/ using document root <none> and password file <none>
I20260924 16:06:45.233876 13983 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:45.233971 13983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:45.234253 13983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:45.236392 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "c41450ec388e425db3ab5b61d093190b"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "319c7d1efe7ee2d22995436be39326c6"
server_key_iv: "6b91169edb4cbc392f369ed33caa863c"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.236958 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "c41450ec388e425db3ab5b61d093190b"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "319c7d1efe7ee2d22995436be39326c6"
server_key_iv: "6b91169edb4cbc392f369ed33caa863c"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.240799 13983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:06:45.243255 13998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.244419 13983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:45.244550 13983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "c41450ec388e425db3ab5b61d093190b"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "319c7d1efe7ee2d22995436be39326c6"
server_key_iv: "6b91169edb4cbc392f369ed33caa863c"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.244714 13983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:45.271425 13983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:45.274721 13983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:45.274950 13983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:45.275575 13983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:45.276461 13983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:45.276543 13983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.276652 13983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:45.276707 13983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.285476 13983 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:40587
I20260924 16:06:45.285497 14112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:40587 every 8 connection(s)
I20260924 16:06:45.286389 13983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:06:45.286988 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 13983
I20260924 16:06:45.287094 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:06:45.287344 13345 external_mini_cluster.cc:1428] Setting key 1bb65734d454c8f803bf6941c9b90cec
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:06:45.301787 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.288243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:35151 (local address 127.13.8.126:36657)
0924 16:06:45.288375 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.288379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.289422 (+ 1043us) server_negotiation.cc:407] Connection header received
0924 16:06:45.290478 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.290482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.290546 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.290638 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:45.292302 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.292856 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.294034 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.294356 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.294477 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.297520 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:45.297525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:45.297530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:45.297567 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:45.299254 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.299877 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.299880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.299881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.299969 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.300380 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.300387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.300392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.300666 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:45.300786 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.301350 (+ 564us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.301514 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:06:45.302747 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.288756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36657 (local address 127.13.8.65:35151)
0924 16:06:45.289265 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.289308 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.290206 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.290825 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.290839 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.291378 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:45.292120 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.292144 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.293002 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.293006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.293888 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.293898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.294515 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.294531 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.294728 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.295348 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:45.295366 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:45.297292 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:45.299381 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.299386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.299398 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.299765 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.300080 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.300083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.300085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.300251 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.300777 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:45.300782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:45.301009 (+ 227us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.301250 (+ 241us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.301556 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":134,"threads_started":1}
I20260924 16:06:45.303956 14113 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36657
I20260924 16:06:45.304220 14113 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:45.304908 14113 heartbeater.cc:511] Master 127.13.8.126:36657 requested a full tablet report, sending...
I20260924 16:06:45.306713 13920 ts_manager.cc:194] Registered new tserver with Master: c41450ec388e425db3ab5b61d093190b (127.13.8.65:40587)
I20260924 16:06:45.307873 13920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:35151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:06:45.317574 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:45.427762 14118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:45.428013 14118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:45.428107 14118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:45.432387 14118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:45.432440 14118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:45.432540 14118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:06:45.437884 14118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:45.438954 14118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:45.440028 14118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:45.449970 14126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.449977 14123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.449996 14124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:45.450031 14118 server_base.cc:1034] running on GCE node
I20260924 16:06:45.450691 14118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:45.451334 14118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:45.452518 14118 hybrid_clock.cc:648] HybridClock initialized: now 1790266005452491 us; error 44 us; skew 500 ppm
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:45.455225 14118 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:06:45.456075 14118 webserver.cc:460] Webserver started at http://127.13.8.66:34979/ using document root <none> and password file <none>
I20260924 16:06:45.456650 14118 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:45.456739 14118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:45.456970 14118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:45.459051 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "3efe6e71f82a45febb535d029e7d9c1f"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "a42c8a1ea84c9e484767a30643f9a7d4"
server_key_iv: "b6fba5df384fe4df56cea11b18fc9ae3"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.459607 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "3efe6e71f82a45febb535d029e7d9c1f"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "a42c8a1ea84c9e484767a30643f9a7d4"
server_key_iv: "b6fba5df384fe4df56cea11b18fc9ae3"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.463290 14118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:06:45.465649 14133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.466897 14118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:45.467027 14118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "3efe6e71f82a45febb535d029e7d9c1f"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "a42c8a1ea84c9e484767a30643f9a7d4"
server_key_iv: "b6fba5df384fe4df56cea11b18fc9ae3"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.467133 14118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:45.481654 14118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:45.484593 14118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:45.484795 14118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:45.485383 14118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:45.486202 14118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:45.486299 14118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.486351 14118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:45.486372 14118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.495083 14118 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:44261
I20260924 16:06:45.495101 14247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:44261 every 8 connection(s)
I20260924 16:06:45.495992 14118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:06:45.503616 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14118
I20260924 16:06:45.503727 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:06:45.504030 13345 external_mini_cluster.cc:1428] Setting key 8e06a0348266b4626d4d892c69d38dfe
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:45.510375 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.497871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:35399 (local address 127.13.8.126:36657)
0924 16:06:45.498005 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.498010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.498919 (+ 909us) server_negotiation.cc:407] Connection header received
0924 16:06:45.499843 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.499847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.499904 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.500001 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:45.501282 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.501869 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.503164 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.503415 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.503523 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.506467 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:45.506470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:45.506472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:45.506495 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:45.508069 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.508703 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.508712 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.508714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.508797 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.509186 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.509190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.509192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.509382 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:45.509503 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.510037 (+ 534us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.510163 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:06:45.511159 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.498312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36657 (local address 127.13.8.66:35399)
0924 16:06:45.498775 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.498818 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.499620 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.500191 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.500198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.500539 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:45.501084 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.501105 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.501984 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.501987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.503026 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.503035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.503582 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.503587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.503806 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.504436 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:45.504458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:45.506259 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:45.508200 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.508209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.508227 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.508578 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.508908 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.508911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.508913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.509073 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.509501 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:45.509509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:45.509769 (+ 260us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.509964 (+ 195us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.510311 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":115,"threads_started":1}
I20260924 16:06:45.512427 14248 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36657
I20260924 16:06:45.512686 14248 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:06:45.513178 14248 heartbeater.cc:511] Master 127.13.8.126:36657 requested a full tablet report, sending...
I20260924 16:06:45.514290 13920 ts_manager.cc:194] Registered new tserver with Master: 3efe6e71f82a45febb535d029e7d9c1f (127.13.8.66:44261)
I20260924 16:06:45.514858 13920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:35399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:06:45.533106 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:45.644631 14253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:45.644891 14253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:45.644933 14253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:45.649374 14253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:45.649430 14253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:45.649489 14253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:06:45.654743 14253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:45.655810 14253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:45.656836 14253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:45.666532 14259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.666499 14258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:45.666522 14261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:45.666743 14253 server_base.cc:1034] running on GCE node
I20260924 16:06:45.667227 14253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:45.667841 14253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:45.669060 14253 hybrid_clock.cc:648] HybridClock initialized: now 1790266005669014 us; error 49 us; skew 500 ppm
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:45.672153 14253 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:06:45.673065 14253 webserver.cc:460] Webserver started at http://127.13.8.67:43375/ using document root <none> and password file <none>
I20260924 16:06:45.673667 14253 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:45.673766 14253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:45.674017 14253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:45.676191 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "6d459cc66160439da71e3db5fea3caf3"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "9e0c3297921349b849ba49879d372c94"
server_key_iv: "d530f8fa2b50f0833cb9728534b26d0b"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.676810 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "6d459cc66160439da71e3db5fea3caf3"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "9e0c3297921349b849ba49879d372c94"
server_key_iv: "d530f8fa2b50f0833cb9728534b26d0b"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.680723 14253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260924 16:06:45.683128 14268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.684336 14253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:45.684468 14253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "6d459cc66160439da71e3db5fea3caf3"
format_stamp: "Formatted at 2026-09-24 16:06:45 on dist-test-slave-vj6n"
server_key: "9e0c3297921349b849ba49879d372c94"
server_key_iv: "d530f8fa2b50f0833cb9728534b26d0b"
server_key_version: "encryptionkey@0"
I20260924 16:06:45.684603 14253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:45.699338 14253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:45.702801 14253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:45.702992 14253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:45.703547 14253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:45.704342 14253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:45.704420 14253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.704492 14253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:45.704536 14253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:45.713223 14253 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:44215
I20260924 16:06:45.713274 14382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:44215 every 8 connection(s)
I20260924 16:06:45.714071 14253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:06:45.719166 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14253
I20260924 16:06:45.719291 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:06:45.719548 13345 external_mini_cluster.cc:1428] Setting key b42618bdb8396392639063adb71d06be
Sep 24 16:06:45 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:45.726764 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.715791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37725 (local address 127.13.8.126:36657)
0924 16:06:45.715937 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.715941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.716748 (+ 807us) server_negotiation.cc:407] Connection header received
0924 16:06:45.717686 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.717690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.717750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.717831 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:45.718972 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.719645 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.721075 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.721304 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.721418 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.723825 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:45.723828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:45.723830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:45.723855 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:45.725032 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.725471 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.725474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.725475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.725545 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.725829 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.725830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.725831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.725989 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:45.726063 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.726561 (+ 498us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.726653 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:06:45.727538 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.716145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36657 (local address 127.13.8.67:37725)
0924 16:06:45.716600 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.716642 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.717418 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.717999 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.718005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.718357 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:45.718822 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.718835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.719759 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.719768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.720949 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.720960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.721457 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.721461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.721637 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.722266 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:45.722294 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:45.723672 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:45.725132 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.725137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.725148 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.725387 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.725622 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:45.725625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:45.725626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:45.725751 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:45.726081 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:45.726086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:45.726358 (+ 272us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.726511 (+ 153us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.726738 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
I20260924 16:06:45.728611 14383 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36657
I20260924 16:06:45.728801 14383 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:45.729215 14383 heartbeater.cc:511] Master 127.13.8.126:36657 requested a full tablet report, sending...
I20260924 16:06:45.730157 13920 ts_manager.cc:194] Registered new tserver with Master: 6d459cc66160439da71e3db5fea3caf3 (127.13.8.67:44215)
I20260924 16:06:45.730738 13920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:37725
I20260924 16:06:45.733115 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:06:45.744875 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.737356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.13.8.126:36657)
0924 16:06:45.737523 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.737529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.737691 (+ 162us) server_negotiation.cc:407] Connection header received
0924 16:06:45.737846 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.737851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.737894 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.737975 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:45.738737 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.739237 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.740083 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.740295 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.740402 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.741715 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:45.741726 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:45.741728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:45.741750 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:45.743209 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.743629 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.743634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.743635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.743730 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:45.743997 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:45.744000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:45.744002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:45.744208 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:45.744345 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.744577 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.744702 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 16:06:45.749786 13920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:06:45.752875 13920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:45.767294 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.13.8.67:44215)
0924 16:06:45.762669 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.762680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.762700 (+ 20us) server_negotiation.cc:407] Connection header received
0924 16:06:45.762783 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.762788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.762959 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.763369 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:45.763514 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764564 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.765900 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.766590 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.766716 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.766868 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.767028 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.767125 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":98,"threads_started":1}
I20260924 16:06:45.767287 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:44215 (local address 127.0.0.1:33276)
0924 16:06:45.762334 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.762346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.762413 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.763106 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.763110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.763132 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:45.763225 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.763231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764696 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.764701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.765752 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.765762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.766716 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.766719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.766822 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.766858 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.766998 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.767132 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":95,"threads_started":1}
I20260924 16:06:45.767588 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:44261 (local address 127.0.0.1:59036)
0924 16:06:45.762016 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.762030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.762108 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.762998 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.763002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.763023 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:45.763149 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.763157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764212 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.764218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.765705 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.765717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.767247 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.767251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.767336 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.767359 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.767409 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.767462 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":22,"thread_start_us":89,"threads_started":1}
I20260924 16:06:45.767596 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.13.8.65:40587)
0924 16:06:45.761881 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.761896 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.761997 (+ 101us) server_negotiation.cc:407] Connection header received
0924 16:06:45.762326 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.762333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.762555 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.762683 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:45.763204 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764402 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.765830 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.766662 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.766780 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.766915 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.767352 (+ 437us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.767453 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":78,"threads_started":1}
I20260924 16:06:45.767738 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40587 (local address 127.0.0.1:36898)
0924 16:06:45.761859 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.761913 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.762109 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.762896 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.762902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.762923 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:45.763073 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.763085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764538 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.764541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.765709 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.765717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.766982 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.766985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.767083 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.767106 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.767183 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.767259 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
I20260924 16:06:45.768338 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.761893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59036 (local address 127.13.8.66:44261)
0924 16:06:45.762336 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.762350 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.762376 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:06:45.762457 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.762462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.762774 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.763036 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:45.763284 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.764082 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.766313 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.767105 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.767860 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.768017 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.768058 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.768162 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":152,"threads_started":1}
I20260924 16:06:45.770738 14046 tablet_service.cc:1467] Processing CreateTablet for tablet a8fd81a320be485b95d4681b7667581f (DEFAULT_TABLE table=test-table [id=1e4a1ad9c728444dabed4cc159f3046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:45.770768 14316 tablet_service.cc:1467] Processing CreateTablet for tablet a8fd81a320be485b95d4681b7667581f (DEFAULT_TABLE table=test-table [id=1e4a1ad9c728444dabed4cc159f3046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:45.770768 14181 tablet_service.cc:1467] Processing CreateTablet for tablet a8fd81a320be485b95d4681b7667581f (DEFAULT_TABLE table=test-table [id=1e4a1ad9c728444dabed4cc159f3046e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:45.771720 14046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8fd81a320be485b95d4681b7667581f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:45.771765 14316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8fd81a320be485b95d4681b7667581f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:45.771765 14181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8fd81a320be485b95d4681b7667581f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:45.778190 14400 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Bootstrap starting.
I20260924 16:06:45.779109 14402 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Bootstrap starting.
I20260924 16:06:45.779726 14401 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Bootstrap starting.
I20260924 16:06:45.780638 14400 tablet_bootstrap.cc:654] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:45.781250 14402 tablet_bootstrap.cc:654] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:45.782490 14401 tablet_bootstrap.cc:654] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:45.783210 14402 log.cc:824] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:45.783365 14401 log.cc:824] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:45.783452 14400 log.cc:824] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:45.785211 14402 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: No bootstrap required, opened a new log
I20260924 16:06:45.785375 14401 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: No bootstrap required, opened a new log
I20260924 16:06:45.785375 14400 tablet_bootstrap.cc:492] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: No bootstrap required, opened a new log
I20260924 16:06:45.785450 14402 ts_tablet_manager.cc:1397] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260924 16:06:45.785562 14401 ts_tablet_manager.cc:1397] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:06:45.785573 14400 ts_tablet_manager.cc:1397] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260924 16:06:45.788687 14402 raft_consensus.cc:348] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.788997 14402 raft_consensus.cc:374] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:45.789052 14402 raft_consensus.cc:729] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d459cc66160439da71e3db5fea3caf3, State: Initialized, Role: FOLLOWER
I20260924 16:06:45.789559 14402 consensus_queue.cc:260] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.789501 14400 raft_consensus.cc:348] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.789501 14401 raft_consensus.cc:348] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.789749 14400 raft_consensus.cc:374] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:45.789749 14401 raft_consensus.cc:374] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:45.789804 14401 raft_consensus.cc:729] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efe6e71f82a45febb535d029e7d9c1f, State: Initialized, Role: FOLLOWER
I20260924 16:06:45.789804 14400 raft_consensus.cc:729] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41450ec388e425db3ab5b61d093190b, State: Initialized, Role: FOLLOWER
I20260924 16:06:45.790323 14383 heartbeater.cc:503] Master 127.13.8.126:36657 was elected leader, sending a full tablet report...
I20260924 16:06:45.790248 14401 consensus_queue.cc:260] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.790643 14402 ts_tablet_manager.cc:1428] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 16:06:45.790964 14248 heartbeater.cc:503] Master 127.13.8.126:36657 was elected leader, sending a full tablet report...
I20260924 16:06:45.791231 14401 ts_tablet_manager.cc:1428] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:06:45.791548 14400 consensus_queue.cc:260] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.792383 14113 heartbeater.cc:503] Master 127.13.8.126:36657 was elected leader, sending a full tablet report...
I20260924 16:06:45.792491 14400 ts_tablet_manager.cc:1428] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260924 16:06:45.962791 14406 raft_consensus.cc:482] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:45.963085 14406 raft_consensus.cc:504] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.964576 14406 leader_election.cc:288] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c41450ec388e425db3ab5b61d093190b (127.13.8.65:40587), 3efe6e71f82a45febb535d029e7d9c1f (127.13.8.66:44261)
W20260924 16:06:45.965003 14384 tablet.cc:2369] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:06:45.968737 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.964963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:53137 (local address 127.13.8.65:40587)
0924 16:06:45.965085 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.965090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.965103 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:45.965306 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.965309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.965380 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.965459 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:45.965830 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.966542 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.967686 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.968121 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.968210 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.968262 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.968523 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.968601 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:06:45.968739 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.964805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40587 (local address 127.13.8.67:53137)
0924 16:06:45.964996 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.965012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.965106 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.965510 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.965513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.965536 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:45.965687 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.965698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.966652 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.966656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.967554 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.967564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.968260 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.968266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.968379 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.968408 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.968481 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.968568 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":36}
I20260924 16:06:45.968766 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.965305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46515 (local address 127.13.8.66:44261)
0924 16:06:45.965429 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:45.965433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:45.965673 (+ 240us) server_negotiation.cc:407] Connection header received
0924 16:06:45.965785 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:45.965790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:45.965847 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:45.965906 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:45.966183 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.966713 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.967819 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.968224 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.968324 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.968387 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:45.968570 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 16:06:45.968653 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:06:45.968760 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:45.964828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:44261 (local address 127.13.8.67:46515)
0924 16:06:45.965571 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:45.965589 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:45.965696 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:45.965940 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:45.965943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:45.965961 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:45.966075 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.966082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.966841 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.966843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.967697 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:45.967707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:45.968345 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:45.968348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:45.968431 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:45.968449 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 16:06:45.968544 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 16:06:45.968605 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":20,"thread_start_us":149,"threads_started":1}
I20260924 16:06:45.969458 14066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8fd81a320be485b95d4681b7667581f" candidate_uuid: "6d459cc66160439da71e3db5fea3caf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c41450ec388e425db3ab5b61d093190b" is_pre_election: true
I20260924 16:06:45.969472 14201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8fd81a320be485b95d4681b7667581f" candidate_uuid: "6d459cc66160439da71e3db5fea3caf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efe6e71f82a45febb535d029e7d9c1f" is_pre_election: true
I20260924 16:06:45.969805 14066 raft_consensus.cc:2454] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d459cc66160439da71e3db5fea3caf3 in term 0.
I20260924 16:06:45.969825 14201 raft_consensus.cc:2454] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d459cc66160439da71e3db5fea3caf3 in term 0.
I20260924 16:06:45.970371 14270 leader_election.cc:302] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3efe6e71f82a45febb535d029e7d9c1f, 6d459cc66160439da71e3db5fea3caf3, c41450ec388e425db3ab5b61d093190b; no voters:
I20260924 16:06:45.970654 14406 raft_consensus.cc:2781] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:45.970726 14406 raft_consensus.cc:482] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:45.970815 14406 raft_consensus.cc:3037] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:45.971930 14406 raft_consensus.cc:504] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.972302 14406 leader_election.cc:288] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [CANDIDATE]: Term 1 election: Requested vote from peers c41450ec388e425db3ab5b61d093190b (127.13.8.65:40587), 3efe6e71f82a45febb535d029e7d9c1f (127.13.8.66:44261)
I20260924 16:06:45.972651 14066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8fd81a320be485b95d4681b7667581f" candidate_uuid: "6d459cc66160439da71e3db5fea3caf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c41450ec388e425db3ab5b61d093190b"
I20260924 16:06:45.972720 14201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8fd81a320be485b95d4681b7667581f" candidate_uuid: "6d459cc66160439da71e3db5fea3caf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efe6e71f82a45febb535d029e7d9c1f"
I20260924 16:06:45.972780 14066 raft_consensus.cc:3037] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:45.972860 14201 raft_consensus.cc:3037] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:45.974002 14066 raft_consensus.cc:2454] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d459cc66160439da71e3db5fea3caf3 in term 1.
I20260924 16:06:45.974466 14269 leader_election.cc:302] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d459cc66160439da71e3db5fea3caf3, c41450ec388e425db3ab5b61d093190b; no voters:
I20260924 16:06:45.974684 14406 raft_consensus.cc:2781] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:45.974948 14406 raft_consensus.cc:686] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [term 1 LEADER]: Becoming Leader. State: Replica: 6d459cc66160439da71e3db5fea3caf3, State: Running, Role: LEADER
I20260924 16:06:45.975299 14406 consensus_queue.cc:237] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } }
I20260924 16:06:45.979030 14201 raft_consensus.cc:2454] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d459cc66160439da71e3db5fea3caf3 in term 1.
I20260924 16:06:45.979732 13919 catalog_manager.cc:5495] T a8fd81a320be485b95d4681b7667581f P 6d459cc66160439da71e3db5fea3caf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d459cc66160439da71e3db5fea3caf3 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "6d459cc66160439da71e3db5fea3caf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41450ec388e425db3ab5b61d093190b" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3efe6e71f82a45febb535d029e7d9c1f" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d459cc66160439da71e3db5fea3caf3" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44215 } health_report { overall_health: HEALTHY } } }
W20260924 16:06:45.997102 14249 tablet.cc:2369] T a8fd81a320be485b95d4681b7667581f P 3efe6e71f82a45febb535d029e7d9c1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:46.020285 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:46.011503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.13.8.65:40587)
0924 16:06:46.011662 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:46.011667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:46.011804 (+ 137us) server_negotiation.cc:407] Connection header received
0924 16:06:46.011928 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:46.011930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:46.011981 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:46.012070 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:46.012701 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.013210 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.014005 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.014311 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.014409 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:46.016716 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:46.016729 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:46.016745 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:46.016768 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:46.018677 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.019046 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.019051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.019052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.019139 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.019352 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.019354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.019355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.019651 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:46.019812 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:46.019952 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 16:06:46.020056 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266006, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260924 16:06:46.037794 14114 tablet.cc:2369] T a8fd81a320be485b95d4681b7667581f P c41450ec388e425db3ab5b61d093190b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266006, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:46.046550 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:46.038471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.13.8.65:40587)
0924 16:06:46.038611 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:46.038616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:46.038795 (+ 179us) server_negotiation.cc:407] Connection header received
0924 16:06:46.038900 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:46.038902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:46.038946 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:46.039015 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:46.039667 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.040173 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.040941 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.041166 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.041246 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:46.043595 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:46.043598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:46.043600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:46.043628 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:46.045015 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.045362 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.045364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.045365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.045434 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.045645 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.045647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.045648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.045843 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:46.045956 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:46.046206 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 16:06:46.046366 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[13875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[13875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:46.071162 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:46.063267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.13.8.65:40587)
0924 16:06:46.063410 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:46.063419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:46.063538 (+ 119us) server_negotiation.cc:407] Connection header received
0924 16:06:46.063651 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:46.063654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:46.063696 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:46.063764 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:46.064394 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.064886 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.065687 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:46.065903 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:46.065994 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:46.068240 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:46.068243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:46.068245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:46.068267 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:46.069634 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.070002 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.070005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.070006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.070079 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:46.070338 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:46.070340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:46.070341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:46.070512 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:46.070602 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:46.070827 (+ 225us) server_negotiation.cc:299] Negotiation successful
0924 16:06:46.070955 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260924 16:06:46.072508 14026 tablet_service.cc:2908] Rejecting scan request for tablet a8fd81a320be485b95d4681b7667581f: Uninitialized: clean time has not yet been initialized
I20260924 16:06:46.074568 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13983
I20260924 16:06:46.080890 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14118
I20260924 16:06:46.086985 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14253
I20260924 16:06:46.092592 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 13888
2026-09-24T16:06:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2469 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[14424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[14424](info): set up 2 sockets
Sep 24 16:06:46 dist-test-slave-vj6n krb5kdc[14424](info): commencing operation
krb5kdc: starting...
W20260924 16:06:47.141417 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:47.260914 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44061
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:44061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:47.372181 14437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:47.372449 14437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:47.372488 14437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:47.376415 14437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:47.376461 14437 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 16:06:47.376477 14437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:47.376494 14437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:47.376510 14437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:47.376525 14437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:47.381724 14437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:44061
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:47.382844 14437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:47.383860 14437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:47.392367 14445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:47.392354 14443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.392491 14437 server_base.cc:1034] running on GCE node
W20260924 16:06:47.392354 14442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.393216 14437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:47.394438 14437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:47.395597 14437 hybrid_clock.cc:648] HybridClock initialized: now 1790266007395580 us; error 29 us; skew 500 ppm
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:47.406574 14437 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:47.407503 14437 webserver.cc:460] Webserver started at http://127.13.8.126:33231/ using document root <none> and password file <none>
I20260924 16:06:47.408082 14437 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:47.408170 14437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:47.408406 14437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:47.410499 14437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "1f544dddebfb4ccab315431a6dcc8739"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "578bea8b692c25cd08a3852727f086f2"
server_key_iv: "24e2bb7e9d8343dae804fa3f9750b286"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.411062 14437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "1f544dddebfb4ccab315431a6dcc8739"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "578bea8b692c25cd08a3852727f086f2"
server_key_iv: "24e2bb7e9d8343dae804fa3f9750b286"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.414769 14437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:06:47.417418 14452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:47.418788 14437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:47.418931 14437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "1f544dddebfb4ccab315431a6dcc8739"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "578bea8b692c25cd08a3852727f086f2"
server_key_iv: "24e2bb7e9d8343dae804fa3f9750b286"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.419046 14437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:47.430335 14437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:47.433681 14437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:47.433938 14437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:47.441759 14437 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:44061
I20260924 16:06:47.441781 14514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:44061 every 8 connection(s)
I20260924 16:06:47.442837 14437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:47.445670 14515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:47.446736 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14437
I20260924 16:06:47.446864 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:47.447155 13345 external_mini_cluster.cc:1428] Setting key 7da1c0a143060fe72289af0d0ddaacd8
I20260924 16:06:47.452064 14515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Bootstrap starting.
I20260924 16:06:47.454406 14515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:47.455161 14515 log.cc:824] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:47.457309 14515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: No bootstrap required, opened a new log
I20260924 16:06:47.460266 14515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } }
I20260924 16:06:47.460515 14515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:47.460611 14515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f544dddebfb4ccab315431a6dcc8739, State: Initialized, Role: FOLLOWER
I20260924 16:06:47.461125 14515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } }
I20260924 16:06:47.461300 14515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:47.461375 14515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:47.461485 14515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:47.461779 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.448623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.13.8.126:44061)
0924 16:06:47.449092 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:47.449107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:47.449143 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:06:47.449768 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:47.449785 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:47.450135 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:47.450548 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:47.451412 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.452460 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.453311 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.453791 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.453961 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.456556 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:47.456563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:47.456583 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:47.456609 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:47.458827 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.459258 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.459264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.459274 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.459379 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.459653 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.459657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.459658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.460081 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:47.460292 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:47.460584 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 16:06:47.460912 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":140,"threads_started":1}
I20260924 16:06:47.462495 14515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } }
I20260924 16:06:47.462889 14515 leader_election.cc:302] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f544dddebfb4ccab315431a6dcc8739; no voters:
I20260924 16:06:47.463215 14515 leader_election.cc:288] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:47.463311 14520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:47.463562 14520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [term 1 LEADER]: Becoming Leader. State: Replica: 1f544dddebfb4ccab315431a6dcc8739, State: Running, Role: LEADER
I20260924 16:06:47.463907 14520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } }
I20260924 16:06:47.464519 14515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:47.465108 14520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f544dddebfb4ccab315431a6dcc8739. Latest consensus state: current_term: 1 leader_uuid: "1f544dddebfb4ccab315431a6dcc8739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } } }
I20260924 16:06:47.465232 14520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:47.466068 14521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f544dddebfb4ccab315431a6dcc8739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f544dddebfb4ccab315431a6dcc8739" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44061 } } }
I20260924 16:06:47.466171 14521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:47.466216 14526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:47.470206 14526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:47.475958 14526 catalog_manager.cc:1347] Generated new cluster ID: b34eed3401fa40ac8d620ecf42195082
I20260924 16:06:47.476037 14526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:47.486617 14526 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:47.487676 14526 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:47.497321 14526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Generated new TSK 0
I20260924 16:06:47.498359 14526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:06:47.537933 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44061
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:47.651173 14542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:47.651468 14542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:47.651540 14542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:47.655534 14542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:47.655608 14542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:47.655674 14542 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:47.655753 14542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:06:47.660914 14542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:47.661952 14542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:47.663043 14542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:47.672869 14548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.672993 14542 server_base.cc:1034] running on GCE node
W20260924 16:06:47.672883 14547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:47.672883 14550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.673543 14542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:47.674188 14542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:47.675405 14542 hybrid_clock.cc:648] HybridClock initialized: now 1790266007675386 us; error 26 us; skew 500 ppm
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:47.678049 14542 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:06:47.679010 14542 webserver.cc:460] Webserver started at http://127.13.8.65:44357/ using document root <none> and password file <none>
I20260924 16:06:47.679587 14542 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:47.679672 14542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:47.679886 14542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:47.681911 14542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "482d32bc986778cc5ff0ebc36ff8829f"
server_key_iv: "b195eebee2064203133d692b2d564d38"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.682518 14542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "482d32bc986778cc5ff0ebc36ff8829f"
server_key_iv: "b195eebee2064203133d692b2d564d38"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.686388 14542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:06:47.688676 14557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:47.689815 14542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:47.689940 14542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "482d32bc986778cc5ff0ebc36ff8829f"
server_key_iv: "b195eebee2064203133d692b2d564d38"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.690044 14542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:47.699381 14542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:47.702157 14542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:47.702360 14542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:47.703781 14542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:47.703835 14542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:47.703894 14542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:47.703925 14542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:47.714462 14542 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:36389
I20260924 16:06:47.714488 14674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:36389 every 8 connection(s)
I20260924 16:06:47.715492 14542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:06:47.717209 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.704433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:40267 (local address 127.13.8.126:44061)
0924 16:06:47.704591 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:47.704598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:47.705858 (+ 1260us) server_negotiation.cc:407] Connection header received
0924 16:06:47.707006 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:47.707010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:47.707066 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:47.707149 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:47.708360 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.708888 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.710118 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.710548 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.710647 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.713438 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:47.713441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:47.713443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:47.713466 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:47.715095 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.715701 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.715705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.715707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.715804 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.716217 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.716224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.716227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.716465 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:47.716575 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:47.716989 (+ 414us) server_negotiation.cc:299] Negotiation successful
0924 16:06:47.717090 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:06:47.718009 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.705270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.65:40267)
0924 16:06:47.705707 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:47.705764 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:47.706805 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:47.707345 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:47.707352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:47.707771 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:47.708217 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.708230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.709011 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.709020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.709968 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.709981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.710803 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.710811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.711053 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.711702 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:47.711718 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:47.713260 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:47.715204 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.715210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.715224 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.715572 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.715927 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.715931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.715933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.716091 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.716569 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:47.716574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:47.716790 (+ 216us) client_negotiation.cc:769] Sending connection context
0924 16:06:47.716941 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 16:06:47.717169 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":129,"threads_started":1}
I20260924 16:06:47.719274 14675 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44061
I20260924 16:06:47.719429 14675 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:47.719935 14675 heartbeater.cc:511] Master 127.13.8.126:44061 requested a full tablet report, sending...
I20260924 16:06:47.721455 14468 ts_manager.cc:194] Registered new tserver with Master: 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389)
I20260924 16:06:47.723050 14468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:40267
I20260924 16:06:47.724210 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14542
I20260924 16:06:47.724327 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:06:47.724630 13345 external_mini_cluster.cc:1428] Setting key 62071896b24d52e675dac1e945d2a8b5
I20260924 16:06:47.728826 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.721383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:60249 (local address 127.13.8.126:44061)
0924 16:06:47.721508 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:47.721513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:47.721684 (+ 171us) server_negotiation.cc:407] Connection header received
0924 16:06:47.721783 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:47.721786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:47.721822 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:47.721925 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:47.722395 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.722885 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.723981 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.724309 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.724411 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.725706 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:47.725709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:47.725712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:47.725734 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:47.727179 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.727681 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.727685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.727687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.727771 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.728036 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.728040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.728042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.728208 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:47.728317 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:47.728562 (+ 245us) server_negotiation.cc:299] Negotiation successful
0924 16:06:47.728669 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:06:47.728838 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.721324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.65:60249)
0924 16:06:47.721597 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:47.721613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:47.721707 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:47.721940 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:47.721943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:47.722152 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:47.722294 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.722301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.723001 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.723005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.723866 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.723875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.724536 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.724541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.724636 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.725196 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:47.725211 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:47.725581 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:47.727287 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.727294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.727299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.727572 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.727860 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.727864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.727867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.727948 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.728307 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:47.728310 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:47.728405 (+ 95us) client_negotiation.cc:769] Sending connection context
0924 16:06:47.728508 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 16:06:47.728645 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:06:47.751181 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44061
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:47.859869 14684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:47.860157 14684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:47.860239 14684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:47.864162 14684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:47.864243 14684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:47.864327 14684 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:47.864411 14684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:06:47.869747 14684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:47.870759 14684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:47.871706 14684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:47.881191 14692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.881242 14684 server_base.cc:1034] running on GCE node
W20260924 16:06:47.881256 14689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:47.881229 14690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:47.881877 14684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:47.882495 14684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:47.883683 14684 hybrid_clock.cc:648] HybridClock initialized: now 1790266007883660 us; error 31 us; skew 500 ppm
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:47.886411 14684 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:06:47.887194 14684 webserver.cc:460] Webserver started at http://127.13.8.66:37993/ using document root <none> and password file <none>
I20260924 16:06:47.887717 14684 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:47.887776 14684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:47.887936 14684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:47.889985 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "b6662c6ff46240ad8ea37741c52ce12b"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "352b09188289755241aabc410e0fc250"
server_key_iv: "7d9d75628427e972b39f4e7e5a20fa6c"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.890697 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "b6662c6ff46240ad8ea37741c52ce12b"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "352b09188289755241aabc410e0fc250"
server_key_iv: "7d9d75628427e972b39f4e7e5a20fa6c"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.895330 14684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:06:47.897715 14699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:47.898931 14684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:47.899048 14684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "b6662c6ff46240ad8ea37741c52ce12b"
format_stamp: "Formatted at 2026-09-24 16:06:47 on dist-test-slave-vj6n"
server_key: "352b09188289755241aabc410e0fc250"
server_key_iv: "7d9d75628427e972b39f4e7e5a20fa6c"
server_key_version: "encryptionkey@0"
I20260924 16:06:47.899160 14684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:47.910337 14684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:47.913381 14684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:47.913586 14684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:47.915059 14684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:47.915125 14684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:47.915194 14684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:47.915256 14684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:06:47 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:47.926564 14684 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:42539
I20260924 16:06:47.927421 14815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:42539 every 8 connection(s)
I20260924 16:06:47.927759 14684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:06:47.929317 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.916030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:43187 (local address 127.13.8.126:44061)
0924 16:06:47.916166 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:47.916170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:47.917263 (+ 1093us) server_negotiation.cc:407] Connection header received
0924 16:06:47.918551 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:47.918555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:47.918622 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:47.918730 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:47.920048 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.920670 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.921850 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.922321 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.922459 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.925520 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:47.925524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:47.925527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:47.925567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:47.927211 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.927802 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.927805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.927807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.927897 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.928227 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.928231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.928233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.928471 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:47.928568 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:47.929038 (+ 470us) server_negotiation.cc:299] Negotiation successful
0924 16:06:47.929156 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:06:47.930116 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.916542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.66:43187)
0924 16:06:47.917115 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:47.917158 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:47.918305 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:47.918907 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:47.918915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:47.919246 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:47.919886 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.919906 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.920817 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.920821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.921714 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.921725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.922510 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.922514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.922704 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.923391 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:47.923417 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:47.925344 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:47.927320 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.927325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.927368 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.927696 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.927989 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.927991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.927993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.928128 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.928570 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:47.928579 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:47.928811 (+ 232us) client_negotiation.cc:769] Sending connection context
0924 16:06:47.928993 (+ 182us) client_negotiation.cc:240] Negotiation successful
0924 16:06:47.929342 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":144,"threads_started":1}
I20260924 16:06:47.931289 14816 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44061
I20260924 16:06:47.931447 14816 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:47.931927 14816 heartbeater.cc:511] Master 127.13.8.126:44061 requested a full tablet report, sending...
I20260924 16:06:47.933017 14469 ts_manager.cc:194] Registered new tserver with Master: b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539)
I20260924 16:06:47.933799 14469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:43187
I20260924 16:06:47.936928 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14684
I20260924 16:06:47.937049 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:06:47.937330 13345 external_mini_cluster.cc:1428] Setting key 1f012332a8a35f786b80966b2425e87a
I20260924 16:06:47.940960 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.933457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:43311 (local address 127.13.8.126:44061)
0924 16:06:47.933579 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:47.933583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:47.933745 (+ 162us) server_negotiation.cc:407] Connection header received
0924 16:06:47.933849 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:47.933852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:47.933898 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:47.933975 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:47.934502 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.935110 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.936174 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.936385 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.936459 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.937653 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:47.937657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:47.937660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:47.937686 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:47.939292 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.939671 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.939677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.939682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.939774 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:47.940023 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:47.940029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:47.940033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:47.940344 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:47.940472 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:47.940760 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 16:06:47.940851 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:06:47.941108 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:47.933364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.66:43311)
0924 16:06:47.933674 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:47.933686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:47.933768 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:47.933987 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:47.933989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:47.934206 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:47.934372 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.934380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.935235 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.935238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.936050 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:47.936062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:47.936485 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:47.936487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:47.936555 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:47.937161 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:47.937175 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:47.937527 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:47.939392 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.939394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.939396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.939578 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.939866 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:47.939868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:47.939869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:47.939940 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:47.940467 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:47.940472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:47.940586 (+ 114us) client_negotiation.cc:769] Sending connection context
0924 16:06:47.940704 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 16:06:47.940876 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:06:47.966048 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44061
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:48.078210 14825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:48.078540 14825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:48.078642 14825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:48.082732 14825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:48.082808 14825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:48.082871 14825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:48.082947 14825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:06:48.088299 14825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:48.089401 14825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:48.090513 14825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:48.100154 14831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:48.100236 14825 server_base.cc:1034] running on GCE node
W20260924 16:06:48.100175 14833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:48.100153 14830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:48.100752 14825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:48.101378 14825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:48.102566 14825 hybrid_clock.cc:648] HybridClock initialized: now 1790266008102535 us; error 39 us; skew 500 ppm
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266008, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:48.105260 14825 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:06:48.106124 14825 webserver.cc:460] Webserver started at http://127.13.8.67:35777/ using document root <none> and password file <none>
I20260924 16:06:48.106704 14825 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:48.106791 14825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:48.107012 14825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:48.108956 14825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "f9ca3e2944e842d3afc859ec8187f81b"
format_stamp: "Formatted at 2026-09-24 16:06:48 on dist-test-slave-vj6n"
server_key: "959fb5005e623da148df8b8795eadd17"
server_key_iv: "ac8e913bdc2436d2e557b331eb29696b"
server_key_version: "encryptionkey@0"
I20260924 16:06:48.109498 14825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "f9ca3e2944e842d3afc859ec8187f81b"
format_stamp: "Formatted at 2026-09-24 16:06:48 on dist-test-slave-vj6n"
server_key: "959fb5005e623da148df8b8795eadd17"
server_key_iv: "ac8e913bdc2436d2e557b331eb29696b"
server_key_version: "encryptionkey@0"
I20260924 16:06:48.113265 14825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:06:48.115794 14841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:48.117033 14825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:06:48.117174 14825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "f9ca3e2944e842d3afc859ec8187f81b"
format_stamp: "Formatted at 2026-09-24 16:06:48 on dist-test-slave-vj6n"
server_key: "959fb5005e623da148df8b8795eadd17"
server_key_iv: "ac8e913bdc2436d2e557b331eb29696b"
server_key_version: "encryptionkey@0"
I20260924 16:06:48.117283 14825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:48.132802 14825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:48.135702 14825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:48.135902 14825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:48.137354 14825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:48.137422 14825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:48.137491 14825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:48.137527 14825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:48.148399 14825 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:34405
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266008, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:48.148468 14956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:34405 every 8 connection(s)
I20260924 16:06:48.149331 14825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:06:48.152081 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 14825
I20260924 16:06:48.152208 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:06:48.152535 13345 external_mini_cluster.cc:1428] Setting key bfb59f2a7448178b62f5a1adbfc0f73d
I20260924 16:06:48.153532 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.138017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:44417 (local address 127.13.8.126:44061)
0924 16:06:48.138150 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.138158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.139904 (+ 1746us) server_negotiation.cc:407] Connection header received
0924 16:06:48.141055 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.141058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.141115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.141222 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.142958 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.143846 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.145703 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.145941 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.146053 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.149146 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.149151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.149154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.149190 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.151161 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.151749 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.151752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.151753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.151870 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.152241 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.152245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.152247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.152503 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.152638 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.153202 (+ 564us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.153332 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:06:48.154502 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.139177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.67:44417)
0924 16:06:48.139756 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.139798 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.140840 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.141452 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.141462 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.141986 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:48.142771 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.142798 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.143981 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.143986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.145536 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.145549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.146136 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.146142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.146414 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.147166 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:48.147206 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:48.148958 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:48.151289 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.151293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.151305 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.151606 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.151974 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.151977 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.151978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.152108 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.152639 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:48.152645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:48.152918 (+ 273us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.153113 (+ 195us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.153432 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":142,"threads_started":1}
I20260924 16:06:48.155583 14957 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44061
I20260924 16:06:48.155725 14957 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:48.156302 14957 heartbeater.cc:511] Master 127.13.8.126:44061 requested a full tablet report, sending...
I20260924 16:06:48.157311 14469 ts_manager.cc:194] Registered new tserver with Master: f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67:34405)
I20260924 16:06:48.158075 14469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:44417
I20260924 16:06:48.164639 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.157943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:33157 (local address 127.13.8.126:44061)
0924 16:06:48.158070 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.158075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.158255 (+ 180us) server_negotiation.cc:407] Connection header received
0924 16:06:48.158411 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.158414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.158460 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.158547 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.159131 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.159589 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.160583 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.160787 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.160859 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.161840 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.161843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.161845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.161864 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.163220 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.163629 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.163631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.163632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.163697 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.163925 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.163927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.163928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.164096 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.164185 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.164458 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.164540 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 16:06:48.164853 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.157824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.13.8.67:33157)
0924 16:06:48.158150 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.158163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.158317 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.158602 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.158605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.158836 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:48.159012 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.159023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.159739 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.159743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.160466 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.160476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.160877 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.160880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.160946 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.161374 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:48.161386 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:48.161720 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:48.163332 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.163335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.163336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.163538 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.163771 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.163773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.163775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.163851 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.164192 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:48.164194 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:48.164267 (+ 73us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.164416 (+ 149us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.164593 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":4,"thread_start_us":122,"threads_started":1}
I20260924 16:06:48.166484 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:48.190572 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.183041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.13.8.126:44061)
0924 16:06:48.183157 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.183160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.183173 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:48.183209 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.183211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.183260 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.183326 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.183980 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.184475 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.185303 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.185499 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.185571 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.187827 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.187830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.187831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.187853 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.189194 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.189563 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.189565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.189566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.189635 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.189893 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.189895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.189896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.190049 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.190143 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.190374 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.190459 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:06:48.193264 14469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:06:48.196429 14469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:48.210736 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.204498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42539 (local address 127.0.0.1:33494)
0924 16:06:48.205126 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.205140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.205224 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.206055 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.206058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.206074 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.206177 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.206183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207735 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.207743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.208778 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.208788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.209903 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.209907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.210328 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210351 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.210424 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.210552 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":325,"thread_start_us":99,"threads_started":1}
I20260924 16:06:48.210798 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.205203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.13.8.66:42539)
0924 16:06:48.205625 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.205635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.205656 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:06:48.205725 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.205730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.205901 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.206385 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.206516 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207575 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.208906 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.209759 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.210054 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210189 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.210523 (+ 334us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.210638 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":111,"threads_started":1}
I20260924 16:06:48.210800 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.205557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.13.8.67:34405)
0924 16:06:48.205956 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.205967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.206204 (+ 237us) server_negotiation.cc:407] Connection header received
0924 16:06:48.206355 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.206359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.206497 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.206645 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.206817 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207546 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.209406 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.209993 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.210092 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210268 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.210578 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.210669 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":88,"threads_started":1}
I20260924 16:06:48.211351 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.204427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36389 (local address 127.0.0.1:41204)
0924 16:06:48.204886 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.204938 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.205059 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.205893 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.205900 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.205925 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.206085 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.206092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207635 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.207641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.208808 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.208818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.209886 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.209890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.210080 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210131 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.211121 (+ 990us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.211189 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":85,"thread_start_us":81,"threads_started":1}
I20260924 16:06:48.211570 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.204582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.13.8.65:36389)
0924 16:06:48.205344 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.205360 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.205387 (+ 27us) server_negotiation.cc:407] Connection header received
0924 16:06:48.205461 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.205468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.205683 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.205887 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.206299 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207511 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.208965 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.209759 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.209881 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210060 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.211282 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.211417 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":65,"threads_started":1}
I20260924 16:06:48.210740 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.205156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:34405 (local address 127.0.0.1:34232)
0924 16:06:48.206098 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.206111 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.206206 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.206624 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.206627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.206644 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.206722 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.206729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.207690 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.207693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.209292 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.209300 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.210328 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.210331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.210426 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.210446 (+ 20us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.210512 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.210561 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":25,"thread_start_us":95,"threads_started":1}
I20260924 16:06:48.214162 14891 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2729742294a848589860f9b4accac (DEFAULT_TABLE table=test-table [id=31d05dff2372479cae37d9b6b24e1b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.214161 14750 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2729742294a848589860f9b4accac (DEFAULT_TABLE table=test-table [id=31d05dff2372479cae37d9b6b24e1b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.214154 14608 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2729742294a848589860f9b4accac (DEFAULT_TABLE table=test-table [id=31d05dff2372479cae37d9b6b24e1b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.215281 14891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2729742294a848589860f9b4accac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.215276 14750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2729742294a848589860f9b4accac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.215301 14608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2729742294a848589860f9b4accac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.223547 14982 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Bootstrap starting.
I20260924 16:06:48.223713 14980 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Bootstrap starting.
I20260924 16:06:48.224404 14981 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Bootstrap starting.
I20260924 16:06:48.225665 14982 tablet_bootstrap.cc:654] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.226210 14980 tablet_bootstrap.cc:654] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.226518 14982 log.cc:824] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:48.226594 14981 tablet_bootstrap.cc:654] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.227250 14980 log.cc:824] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:48.227550 14981 log.cc:824] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:48.228463 14982 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: No bootstrap required, opened a new log
I20260924 16:06:48.228658 14982 ts_tablet_manager.cc:1397] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260924 16:06:48.229424 14980 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: No bootstrap required, opened a new log
I20260924 16:06:48.229429 14981 tablet_bootstrap.cc:492] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: No bootstrap required, opened a new log
I20260924 16:06:48.229635 14981 ts_tablet_manager.cc:1397] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:06:48.229635 14980 ts_tablet_manager.cc:1397] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260924 16:06:48.232156 14982 raft_consensus.cc:348] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.232373 14982 raft_consensus.cc:374] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.232406 14982 raft_consensus.cc:729] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6662c6ff46240ad8ea37741c52ce12b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.232818 14982 consensus_queue.cc:260] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.233398 14816 heartbeater.cc:503] Master 127.13.8.126:44061 was elected leader, sending a full tablet report...
I20260924 16:06:48.233777 14982 ts_tablet_manager.cc:1428] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260924 16:06:48.233989 14981 raft_consensus.cc:348] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.233989 14980 raft_consensus.cc:348] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.234264 14980 raft_consensus.cc:374] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.234246 14981 raft_consensus.cc:374] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.234324 14980 raft_consensus.cc:729] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2838cc6aa8a04f4ca151e32d8cd6b5ea, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.234334 14981 raft_consensus.cc:729] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ca3e2944e842d3afc859ec8187f81b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.234885 14980 consensus_queue.cc:260] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.234885 14981 consensus_queue.cc:260] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.235920 14981 ts_tablet_manager.cc:1428] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260924 16:06:48.235920 14980 ts_tablet_manager.cc:1428] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:06:48.236258 14675 heartbeater.cc:503] Master 127.13.8.126:44061 was elected leader, sending a full tablet report...
I20260924 16:06:48.236280 14957 heartbeater.cc:503] Master 127.13.8.126:44061 was elected leader, sending a full tablet report...
I20260924 16:06:48.339310 14986 raft_consensus.cc:482] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:48.339535 14986 raft_consensus.cc:504] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.340775 14986 leader_election.cc:288] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67:34405)
I20260924 16:06:48.344324 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.340987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36389 (local address 127.13.8.66:37057)
0924 16:06:48.341123 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.341141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.341244 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.341552 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.341555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.341575 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.341679 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.341684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.342488 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.342494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.343310 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.343318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.343916 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.343920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.344016 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.344045 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.344126 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.344198 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260924 16:06:48.344324 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.341106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:37057 (local address 127.13.8.65:36389)
0924 16:06:48.341263 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.341268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.341287 (+ 19us) server_negotiation.cc:407] Connection header received
0924 16:06:48.341346 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.341351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.341414 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.341496 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.341791 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.342368 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.343430 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.343808 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.343913 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.343971 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.344144 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.344205 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:06:48.345113 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.341271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:34405 (local address 127.13.8.66:53613)
0924 16:06:48.341605 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.341618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.341742 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.341984 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.341987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.342002 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.342131 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.342136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.342904 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.342907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.344041 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.344050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.344808 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.344811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.344881 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.344898 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.344956 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.345019 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260924 16:06:48.345018 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2729742294a848589860f9b4accac" candidate_uuid: "b6662c6ff46240ad8ea37741c52ce12b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" is_pre_election: true
I20260924 16:06:48.345220 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.341449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53613 (local address 127.13.8.67:34405)
0924 16:06:48.341579 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.341583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.341703 (+ 120us) server_negotiation.cc:407] Connection header received
0924 16:06:48.341843 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.341846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.341891 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.341973 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.342252 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.342793 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.344169 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.344682 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.344788 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.344841 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.345017 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.345091 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:06:48.345378 14628 raft_consensus.cc:2454] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6662c6ff46240ad8ea37741c52ce12b in term 0.
I20260924 16:06:48.345752 14911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2729742294a848589860f9b4accac" candidate_uuid: "b6662c6ff46240ad8ea37741c52ce12b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ca3e2944e842d3afc859ec8187f81b" is_pre_election: true
I20260924 16:06:48.345884 14700 leader_election.cc:302] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2838cc6aa8a04f4ca151e32d8cd6b5ea, b6662c6ff46240ad8ea37741c52ce12b; no voters:
I20260924 16:06:48.346035 14911 raft_consensus.cc:2454] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6662c6ff46240ad8ea37741c52ce12b in term 0.
I20260924 16:06:48.346159 14986 raft_consensus.cc:2781] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:48.346292 14986 raft_consensus.cc:482] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:48.346357 14986 raft_consensus.cc:3037] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.347375 14986 raft_consensus.cc:504] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.347709 14986 leader_election.cc:288] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [CANDIDATE]: Term 1 election: Requested vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67:34405)
I20260924 16:06:48.348039 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2729742294a848589860f9b4accac" candidate_uuid: "b6662c6ff46240ad8ea37741c52ce12b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
I20260924 16:06:48.348135 14911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2729742294a848589860f9b4accac" candidate_uuid: "b6662c6ff46240ad8ea37741c52ce12b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ca3e2944e842d3afc859ec8187f81b"
I20260924 16:06:48.348172 14628 raft_consensus.cc:3037] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.348263 14911 raft_consensus.cc:3037] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.349207 14628 raft_consensus.cc:2454] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6662c6ff46240ad8ea37741c52ce12b in term 1.
I20260924 16:06:48.349550 14700 leader_election.cc:302] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2838cc6aa8a04f4ca151e32d8cd6b5ea, b6662c6ff46240ad8ea37741c52ce12b; no voters:
I20260924 16:06:48.349681 14911 raft_consensus.cc:2454] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6662c6ff46240ad8ea37741c52ce12b in term 1.
I20260924 16:06:48.349756 14986 raft_consensus.cc:2781] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:48.350010 14986 raft_consensus.cc:686] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 1 LEADER]: Becoming Leader. State: Replica: b6662c6ff46240ad8ea37741c52ce12b, State: Running, Role: LEADER
I20260924 16:06:48.350406 14986 consensus_queue.cc:237] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.353438 14468 catalog_manager.cc:5495] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b reported cstate change: term changed from 0 to 1, leader changed from <none> to b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66). New cstate: current_term: 1 leader_uuid: "b6662c6ff46240ad8ea37741c52ce12b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260924 16:06:48.368620 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.365126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.13.8.66:42539)
0924 16:06:48.365242 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.365245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.365281 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:06:48.365478 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.365480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.365547 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.365678 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:48.365924 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.366456 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.367371 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.367578 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.367655 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.367914 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.368318 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.368403 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.368468 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.368516 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:06:48.374148 14911 raft_consensus.cc:1261] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Refusing update from remote peer b6662c6ff46240ad8ea37741c52ce12b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.374184 14628 raft_consensus.cc:1261] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Refusing update from remote peer b6662c6ff46240ad8ea37741c52ce12b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.374933 14986 consensus_queue.cc:1035] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.375161 14990 consensus_queue.cc:1035] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.383477 14994 mvcc.cc:205] Tried to move back new op lower bound from 7332929570293874688 to 7332929570205765632. Current Snapshot: MvccSnapshot[applied={T|T < 7332929570293874688}]
I20260924 16:06:48.384047 14995 mvcc.cc:205] Tried to move back new op lower bound from 7332929570293874688 to 7332929570205765632. Current Snapshot: MvccSnapshot[applied={T|T < 7332929570293874688}]
I20260924 16:06:48.393689 14469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:48.393921 14469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:48.396481 14469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Sending DeleteTablet for 3 replicas of tablet 2ef2729742294a848589860f9b4accac
I20260924 16:06:48.397343 14891 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2729742294a848589860f9b4accac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:34232
I20260924 16:06:48.397326 14750 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2729742294a848589860f9b4accac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:33494
I20260924 16:06:48.397830 14608 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2729742294a848589860f9b4accac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41204
I20260924 16:06:48.398402 15005 tablet_replica.cc:330] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: stopping tablet replica
I20260924 16:06:48.398422 15004 tablet_replica.cc:330] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: stopping tablet replica
I20260924 16:06:48.398741 15003 tablet_replica.cc:330] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: stopping tablet replica
I20260924 16:06:48.398788 15004 raft_consensus.cc:2229] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:48.398723 14469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 16:06:48.399020 15003 raft_consensus.cc:2229] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 1 LEADER]: Raft consensus shutting down.
W20260924 16:06:48.399139 14469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:48.399147 15004 raft_consensus.cc:2258] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.399394 15003 raft_consensus.cc:2258] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.399883 15005 raft_consensus.cc:2229] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:48.401065 15003 ts_tablet_manager.cc:1905] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.401455 15005 raft_consensus.cc:2258] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.402132 15004 ts_tablet_manager.cc:1905] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.403828 15005 ts_tablet_manager.cc:1905] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.404131 15003 ts_tablet_manager.cc:1918] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:06:48.404233 15003 log.cc:1196] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/2ef2729742294a848589860f9b4accac
I20260924 16:06:48.404572 15003 ts_tablet_manager.cc:1939] T 2ef2729742294a848589860f9b4accac P b6662c6ff46240ad8ea37741c52ce12b: Deleting consensus metadata
I20260924 16:06:48.405473 14750 tablet_service.cc:1467] Processing CreateTablet for tablet a944d0a21b654e4a984111afc1537db4 (DEFAULT_TABLE table=test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.405736 14750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a944d0a21b654e4a984111afc1537db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.406234 14608 tablet_service.cc:1467] Processing CreateTablet for tablet a944d0a21b654e4a984111afc1537db4 (DEFAULT_TABLE table=test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.406383 14891 tablet_service.cc:1467] Processing CreateTablet for tablet a944d0a21b654e4a984111afc1537db4 (DEFAULT_TABLE table=test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:48.406498 14608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a944d0a21b654e4a984111afc1537db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.406611 14891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a944d0a21b654e4a984111afc1537db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.407744 14454 catalog_manager.cc:4841] TS b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539): tablet 2ef2729742294a848589860f9b4accac (table test-table [id=31d05dff2372479cae37d9b6b24e1b1a]) successfully deleted
I20260924 16:06:48.408175 14982 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Bootstrap starting.
I20260924 16:06:48.408880 14980 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Bootstrap starting.
I20260924 16:06:48.409276 14982 tablet_bootstrap.cc:654] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.409807 14980 tablet_bootstrap.cc:654] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.411163 14981 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Bootstrap starting.
I20260924 16:06:48.411266 14980 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: No bootstrap required, opened a new log
I20260924 16:06:48.411347 14980 ts_tablet_manager.cc:1397] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:06:48.412153 14981 tablet_bootstrap.cc:654] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.412168 14980 raft_consensus.cc:348] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.412310 14980 raft_consensus.cc:374] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.412352 14980 raft_consensus.cc:729] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2838cc6aa8a04f4ca151e32d8cd6b5ea, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.412485 14980 consensus_queue.cc:260] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.412550 14982 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: No bootstrap required, opened a new log
I20260924 16:06:48.412622 14982 ts_tablet_manager.cc:1397] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260924 16:06:48.412781 14980 ts_tablet_manager.cc:1428] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260924 16:06:48.413113 14982 raft_consensus.cc:348] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.413226 14982 raft_consensus.cc:374] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.413265 14982 raft_consensus.cc:729] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6662c6ff46240ad8ea37741c52ce12b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.413390 14982 consensus_queue.cc:260] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.414289 15005 ts_tablet_manager.cc:1918] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:06:48.413168 15004 ts_tablet_manager.cc:1918] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:06:48.414490 15004 log.cc:1196] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/2ef2729742294a848589860f9b4accac
I20260924 16:06:48.414568 14982 ts_tablet_manager.cc:1428] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 16:06:48.414904 15004 ts_tablet_manager.cc:1939] T 2ef2729742294a848589860f9b4accac P f9ca3e2944e842d3afc859ec8187f81b: Deleting consensus metadata
I20260924 16:06:48.415377 15005 log.cc:1196] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/2ef2729742294a848589860f9b4accac
I20260924 16:06:48.415752 15005 ts_tablet_manager.cc:1939] T 2ef2729742294a848589860f9b4accac P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting consensus metadata
I20260924 16:06:48.416422 14453 catalog_manager.cc:4841] TS f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67:34405): tablet 2ef2729742294a848589860f9b4accac (table test-table [id=31d05dff2372479cae37d9b6b24e1b1a]) successfully deleted
I20260924 16:06:48.416996 14453 catalog_manager.cc:4841] TS 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389): tablet 2ef2729742294a848589860f9b4accac (table test-table [id=31d05dff2372479cae37d9b6b24e1b1a]) successfully deleted
I20260924 16:06:48.417333 14981 tablet_bootstrap.cc:492] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: No bootstrap required, opened a new log
I20260924 16:06:48.417435 14981 ts_tablet_manager.cc:1397] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260924 16:06:48.417924 14981 raft_consensus.cc:348] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.418042 14981 raft_consensus.cc:374] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.418087 14981 raft_consensus.cc:729] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ca3e2944e842d3afc859ec8187f81b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.418244 14981 consensus_queue.cc:260] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.418520 14981 ts_tablet_manager.cc:1428] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260924 16:06:48.429448 14817 tablet.cc:2369] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:06:48.466826 14676 tablet.cc:2369] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:06:48.572127 14987 raft_consensus.cc:482] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:48.572289 14987 raft_consensus.cc:504] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.573351 14987 leader_election.cc:288] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539)
I20260924 16:06:48.578347 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.573679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:36273 (local address 127.13.8.65:36389)
0924 16:06:48.573792 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.573799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.573878 (+ 79us) server_negotiation.cc:407] Connection header received
0924 16:06:48.574003 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.574005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.574069 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.574157 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.574505 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.575306 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577039 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.577677 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577782 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.577836 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.578137 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.578203 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:06:48.578347 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.573715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:41307 (local address 127.13.8.66:42539)
0924 16:06:48.573844 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.573851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.574093 (+ 242us) server_negotiation.cc:407] Connection header received
0924 16:06:48.574158 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.574160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.574203 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.574300 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:48.574527 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.575306 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577039 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.577645 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577769 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.577838 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.578137 (+ 299us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.578203 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:06:48.578377 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.573557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36389 (local address 127.13.8.67:36273)
0924 16:06:48.573767 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.573782 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.573899 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.574192 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.574194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.574206 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.574372 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.574380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.575423 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.575427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.576906 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.576918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.577805 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.577956 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.578030 (+ 74us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.578106 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.578205 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":148}
I20260924 16:06:48.578388 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.573616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42539 (local address 127.13.8.67:41307)
0924 16:06:48.573986 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.574000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.574082 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.574304 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.574307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.574319 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:48.574424 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.574432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.575423 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.575427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.576913 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.576918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.577816 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.577819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.577981 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.578038 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.578106 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.578201 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":223,"thread_start_us":101,"threads_started":1}
I20260924 16:06:48.578886 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a944d0a21b654e4a984111afc1537db4" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" is_pre_election: true
I20260924 16:06:48.579048 14628 raft_consensus.cc:2454] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 0.
I20260924 16:06:48.579293 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a944d0a21b654e4a984111afc1537db4" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6662c6ff46240ad8ea37741c52ce12b" is_pre_election: true
I20260924 16:06:48.579429 14842 leader_election.cc:302] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2838cc6aa8a04f4ca151e32d8cd6b5ea, f9ca3e2944e842d3afc859ec8187f81b; no voters:
I20260924 16:06:48.579531 14769 raft_consensus.cc:2454] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 0.
I20260924 16:06:48.579689 14987 raft_consensus.cc:2781] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:48.579782 14987 raft_consensus.cc:482] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:48.579829 14987 raft_consensus.cc:3037] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.580974 14987 raft_consensus.cc:504] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.581341 14987 leader_election.cc:288] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 election: Requested vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539)
I20260924 16:06:48.581686 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a944d0a21b654e4a984111afc1537db4" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6662c6ff46240ad8ea37741c52ce12b"
I20260924 16:06:48.581698 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a944d0a21b654e4a984111afc1537db4" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
I20260924 16:06:48.581818 14769 raft_consensus.cc:3037] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.581856 14628 raft_consensus.cc:3037] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.582856 14628 raft_consensus.cc:2454] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 1.
I20260924 16:06:48.582865 14769 raft_consensus.cc:2454] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 1.
I20260924 16:06:48.583144 14842 leader_election.cc:302] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2838cc6aa8a04f4ca151e32d8cd6b5ea, f9ca3e2944e842d3afc859ec8187f81b; no voters:
I20260924 16:06:48.583309 14987 raft_consensus.cc:2781] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:48.583598 14987 raft_consensus.cc:686] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 1 LEADER]: Becoming Leader. State: Replica: f9ca3e2944e842d3afc859ec8187f81b, State: Running, Role: LEADER
I20260924 16:06:48.583864 14987 consensus_queue.cc:237] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.586282 14469 catalog_manager.cc:5495] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b reported cstate change: term changed from 0 to 1, leader changed from <none> to f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67). New cstate: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } health_report { overall_health: HEALTHY } } }
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:48.628603 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.620113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44061 (local address 127.0.0.1:51500)
0924 16:06:48.620406 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.620423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.620576 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.620823 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.620826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.621318 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:48.621416 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.621423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.622177 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.622184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.622821 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.622831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.623269 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.623273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.623433 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.623995 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:48.624007 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:48.625462 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:48.626976 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.626989 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.626996 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.627237 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.627509 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:48.627512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:48.627514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:48.627591 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:48.627969 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:48.627977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:48.628092 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.628207 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.628454 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":132,"thread_start_us":102,"threads_started":1}
I20260924 16:06:48.628603 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.620164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.13.8.126:44061)
0924 16:06:48.620295 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.620301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.620541 (+ 240us) server_negotiation.cc:407] Connection header received
0924 16:06:48.620667 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.620670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.620714 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.620804 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.621552 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.622072 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.622939 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.623151 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.623256 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.625598 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.625602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.625604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.625630 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.626875 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.627332 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.627339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.627343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.627416 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.627682 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.627685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.627687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.627865 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.627969 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.628253 (+ 284us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.628385 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":42}
I20260924 16:06:48.632741 14469 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 16:06:48.633406 14469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:48.638046 14750 tablet_service.cc:1467] Processing CreateTablet for tablet 896c50433cc64067bfa5692747166dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf1022da69dd4ea8a98c6d76da5be320]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:48.638051 14608 tablet_service.cc:1467] Processing CreateTablet for tablet 896c50433cc64067bfa5692747166dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf1022da69dd4ea8a98c6d76da5be320]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:48.638187 14891 tablet_service.cc:1467] Processing CreateTablet for tablet 896c50433cc64067bfa5692747166dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf1022da69dd4ea8a98c6d76da5be320]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:48.638468 14750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896c50433cc64067bfa5692747166dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.638487 14608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896c50433cc64067bfa5692747166dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.638528 14891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896c50433cc64067bfa5692747166dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:48.640790 14982 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: Bootstrap starting.
I20260924 16:06:48.641338 14981 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: Bootstrap starting.
I20260924 16:06:48.641338 14980 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Bootstrap starting.
I20260924 16:06:48.641666 14982 tablet_bootstrap.cc:654] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.642330 14980 tablet_bootstrap.cc:654] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.642330 14981 tablet_bootstrap.cc:654] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:48.643633 14982 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: No bootstrap required, opened a new log
I20260924 16:06:48.643653 14981 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: No bootstrap required, opened a new log
I20260924 16:06:48.643743 14982 ts_tablet_manager.cc:1397] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260924 16:06:48.643756 14981 ts_tablet_manager.cc:1397] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 16:06:48.644114 14982 raft_consensus.cc:348] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.644230 14982 raft_consensus.cc:374] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.644274 14982 raft_consensus.cc:729] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6662c6ff46240ad8ea37741c52ce12b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.644408 14982 consensus_queue.cc:260] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.644574 14981 raft_consensus.cc:348] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.644688 14981 raft_consensus.cc:374] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.644702 14982 ts_tablet_manager.cc:1428] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:06:48.644644 14990 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.644721 14981 raft_consensus.cc:729] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ca3e2944e842d3afc859ec8187f81b, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.644773 14990 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.644845 14981 consensus_queue.cc:260] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.645061 14987 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.645224 14987 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.645143 14981 ts_tablet_manager.cc:1428] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:06:48.645937 14980 tablet_bootstrap.cc:492] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: No bootstrap required, opened a new log
I20260924 16:06:48.646013 14980 ts_tablet_manager.cc:1397] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260924 16:06:48.646512 14980 raft_consensus.cc:348] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.646632 14980 raft_consensus.cc:374] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:48.646682 14980 raft_consensus.cc:729] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2838cc6aa8a04f4ca151e32d8cd6b5ea, State: Initialized, Role: FOLLOWER
I20260924 16:06:48.646821 14980 consensus_queue.cc:260] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.647080 14988 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.647203 14988 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.647578 14980 ts_tablet_manager.cc:1428] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260924 16:06:48.650911 14958 tablet.cc:2369] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:06:48.692237 14987 raft_consensus.cc:482] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:48.692373 14987 raft_consensus.cc:504] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.692867 14987 leader_election.cc:288] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539)
I20260924 16:06:48.693230 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896c50433cc64067bfa5692747166dea" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" is_pre_election: true
I20260924 16:06:48.693310 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896c50433cc64067bfa5692747166dea" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6662c6ff46240ad8ea37741c52ce12b" is_pre_election: true
I20260924 16:06:48.693423 14628 raft_consensus.cc:2454] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 0.
I20260924 16:06:48.693488 14769 raft_consensus.cc:2454] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 0.
I20260924 16:06:48.693809 14842 leader_election.cc:302] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2838cc6aa8a04f4ca151e32d8cd6b5ea, f9ca3e2944e842d3afc859ec8187f81b; no voters:
I20260924 16:06:48.694006 14987 raft_consensus.cc:2781] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:48.694069 14987 raft_consensus.cc:482] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:48.694123 14987 raft_consensus.cc:3037] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.695199 14987 raft_consensus.cc:504] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.695528 14987 leader_election.cc:288] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 election: Requested vote from peers 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389), b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539)
I20260924 16:06:48.695874 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896c50433cc64067bfa5692747166dea" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6662c6ff46240ad8ea37741c52ce12b"
I20260924 16:06:48.695874 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896c50433cc64067bfa5692747166dea" candidate_uuid: "f9ca3e2944e842d3afc859ec8187f81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea"
I20260924 16:06:48.695988 14628 raft_consensus.cc:3037] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.695988 14769 raft_consensus.cc:3037] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:48.697070 14769 raft_consensus.cc:2454] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 1.
I20260924 16:06:48.697067 14628 raft_consensus.cc:2454] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ca3e2944e842d3afc859ec8187f81b in term 1.
I20260924 16:06:48.697358 14843 leader_election.cc:302] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6662c6ff46240ad8ea37741c52ce12b, f9ca3e2944e842d3afc859ec8187f81b; no voters:
I20260924 16:06:48.697556 14987 raft_consensus.cc:2781] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:48.697655 14987 raft_consensus.cc:686] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [term 1 LEADER]: Becoming Leader. State: Replica: f9ca3e2944e842d3afc859ec8187f81b, State: Running, Role: LEADER
I20260924 16:06:48.697801 14987 consensus_queue.cc:237] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } }
I20260924 16:06:48.698455 15010 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: TxnStatusTablet state changed. Reason: New leader f9ca3e2944e842d3afc859ec8187f81b. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.698617 15010 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:48.698870 15022 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 16:06:48.699046 15022 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 16:06:48.699431 14468 catalog_manager.cc:5495] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b reported cstate change: term changed from 0 to 1, leader changed from <none> to f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67). New cstate: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } health_report { overall_health: HEALTHY } } }
I20260924 16:06:48.715683 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.711373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.13.8.67:34405)
0924 16:06:48.711489 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.711492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.711505 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:48.711700 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.711703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.711760 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.711853 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:48.712131 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.712707 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.713924 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.714150 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.714299 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.714726 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.715286 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.715394 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.715496 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.715555 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:06:48.715704 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.711281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:34405 (local address 127.0.0.1:34236)
0924 16:06:48.711423 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.711438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.711538 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.711888 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.711891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.711902 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:48.712015 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.712021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.712826 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.712829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.713819 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.713831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.714409 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.714413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.714506 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.714562 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.715383 (+ 821us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.715399 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.715475 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.715564 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 16:06:48.718971 14628 raft_consensus.cc:1261] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Refusing update from remote peer f9ca3e2944e842d3afc859ec8187f81b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.719167 14769 raft_consensus.cc:1261] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Refusing update from remote peer f9ca3e2944e842d3afc859ec8187f81b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.719590 15010 consensus_queue.cc:1035] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.719846 14987 consensus_queue.cc:1035] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.721400 14988 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: TxnStatusTablet state changed. Reason: New leader f9ca3e2944e842d3afc859ec8187f81b. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.721530 14988 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.721472 14990 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: TxnStatusTablet state changed. Reason: New leader f9ca3e2944e842d3afc859ec8187f81b. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.721575 14990 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.723667 14987 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.723778 14987 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:48.723886 14987 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.723943 14987 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P f9ca3e2944e842d3afc859ec8187f81b: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:48.724548 14988 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.724680 14988 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P 2838cc6aa8a04f4ca151e32d8cd6b5ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.725769 14993 mvcc.cc:205] Tried to move back new op lower bound from 7332929571707711488 to 7332929571627421696. Current Snapshot: MvccSnapshot[applied={T|T < 7332929571707711488}]
I20260924 16:06:48.725925 14990 tablet_replica.cc:439] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f9ca3e2944e842d3afc859ec8187f81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 } } peers { permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 } } peers { permanent_uuid: "f9ca3e2944e842d3afc859ec8187f81b" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 34405 } } }
I20260924 16:06:48.726018 14990 tablet_replica.cc:442] T 896c50433cc64067bfa5692747166dea P b6662c6ff46240ad8ea37741c52ce12b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:48.733330 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.729633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.13.8.67:34405)
0924 16:06:48.729746 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.729750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.729765 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:06:48.729840 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.729843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.729907 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.729986 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:48.730214 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.731012 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.732164 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.732542 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.732646 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.732895 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.732978 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.733051 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.733143 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.733195 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:06:48.744753 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.741605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:45929 (local address 127.13.8.67:34405)
0924 16:06:48.741697 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.741701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.741805 (+ 104us) server_negotiation.cc:407] Connection header received
0924 16:06:48.741936 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.741941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.741985 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.742057 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:48.742348 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.742829 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.743730 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.743934 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.744038 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.744310 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.744382 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.744459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.744552 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.744605 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":32}
I20260924 16:06:48.744761 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.741354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:34405 (local address 127.13.8.67:45929)
0924 16:06:48.741722 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:48.741736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:48.741843 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:48.742090 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:48.742093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:48.742112 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:48.742259 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.742266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.742940 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.742944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.743611 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:48.743620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.744039 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.744042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:48.744130 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.744179 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:48.744466 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:48.744475 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 16:06:48.744541 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 16:06:48.744622 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260924 16:06:48.752036 14628 raft_consensus.cc:1261] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Refusing update from remote peer f9ca3e2944e842d3afc859ec8187f81b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.752393 14769 raft_consensus.cc:1261] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Refusing update from remote peer f9ca3e2944e842d3afc859ec8187f81b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:48.752452 15027 consensus_queue.cc:1035] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2838cc6aa8a04f4ca151e32d8cd6b5ea" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.752774 15027 consensus_queue.cc:1035] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6662c6ff46240ad8ea37741c52ce12b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:48.791949 14468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:48.792083 14468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:48.794024 14468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f544dddebfb4ccab315431a6dcc8739: Sending DeleteTablet for 3 replicas of tablet a944d0a21b654e4a984111afc1537db4
I20260924 16:06:48.794544 14608 tablet_service.cc:1514] Processing DeleteTablet for tablet a944d0a21b654e4a984111afc1537db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41204
I20260924 16:06:48.794636 14750 tablet_service.cc:1514] Processing DeleteTablet for tablet a944d0a21b654e4a984111afc1537db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:33494
I20260924 16:06:48.794757 15005 tablet_replica.cc:330] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: stopping tablet replica
I20260924 16:06:48.794817 15003 tablet_replica.cc:330] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: stopping tablet replica
I20260924 16:06:48.794881 15005 raft_consensus.cc:2229] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:48.794921 15003 raft_consensus.cc:2229] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:48.795004 15005 raft_consensus.cc:2258] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.795025 15003 raft_consensus.cc:2258] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.796571 15003 ts_tablet_manager.cc:1905] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.796774 15005 ts_tablet_manager.cc:1905] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.799198 15005 ts_tablet_manager.cc:1918] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:48.799286 15005 log.cc:1196] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/a944d0a21b654e4a984111afc1537db4
I20260924 16:06:48.799556 15005 ts_tablet_manager.cc:1939] T a944d0a21b654e4a984111afc1537db4 P 2838cc6aa8a04f4ca151e32d8cd6b5ea: Deleting consensus metadata
I20260924 16:06:48.799733 15003 ts_tablet_manager.cc:1918] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:48.799811 15003 log.cc:1196] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/a944d0a21b654e4a984111afc1537db4
I20260924 16:06:48.800071 15003 ts_tablet_manager.cc:1939] T a944d0a21b654e4a984111afc1537db4 P b6662c6ff46240ad8ea37741c52ce12b: Deleting consensus metadata
I20260924 16:06:48.800272 14453 catalog_manager.cc:4841] TS 2838cc6aa8a04f4ca151e32d8cd6b5ea (127.13.8.65:36389): tablet a944d0a21b654e4a984111afc1537db4 (table test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]) successfully deleted
I20260924 16:06:48.800606 14890 tablet_service.cc:1514] Processing DeleteTablet for tablet a944d0a21b654e4a984111afc1537db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:34232
I20260924 16:06:48.800710 14454 catalog_manager.cc:4841] TS b6662c6ff46240ad8ea37741c52ce12b (127.13.8.66:42539): tablet a944d0a21b654e4a984111afc1537db4 (table test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]) successfully deleted
I20260924 16:06:48.800782 15004 tablet_replica.cc:330] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: stopping tablet replica
I20260924 16:06:48.800892 15004 raft_consensus.cc:2229] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:06:48.801144 15004 raft_consensus.cc:2258] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:48.801777 15004 ts_tablet_manager.cc:1905] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:48.804661 15004 ts_tablet_manager.cc:1918] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:48.804764 15004 log.cc:1196] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/a944d0a21b654e4a984111afc1537db4
I20260924 16:06:48.805064 15004 ts_tablet_manager.cc:1939] T a944d0a21b654e4a984111afc1537db4 P f9ca3e2944e842d3afc859ec8187f81b: Deleting consensus metadata
I20260924 16:06:48.805721 14453 catalog_manager.cc:4841] TS f9ca3e2944e842d3afc859ec8187f81b (127.13.8.67:34405): tablet a944d0a21b654e4a984111afc1537db4 (table test-table [id=c35a11d5f1eb41a686d54d0e81fbc928]) successfully deleted
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[14424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:06:48.809940 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.796277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.13.8.65:36389)
0924 16:06:48.798372 (+ 2095us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.798376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.798680 (+ 304us) server_negotiation.cc:407] Connection header received
0924 16:06:48.798867 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.798870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.798919 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.798973 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.800331 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.800967 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.802163 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.802524 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.802620 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.806355 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.806366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.806383 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.806407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.808221 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.808609 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.808616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.808618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.808706 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.808950 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.808954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.808955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.809268 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.809478 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.809617 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.809734 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2016}
W20260924 16:06:48.811329 14669 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41208
I20260924 16:06:48.820338 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:48.812858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.13.8.126:44061)
0924 16:06:48.812971 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:48.812975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:48.813068 (+ 93us) server_negotiation.cc:407] Connection header received
0924 16:06:48.813160 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:48.813163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:48.813211 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:48.813295 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:48.813879 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.814708 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.815464 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:48.815788 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:48.815890 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:48.817078 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:48.817081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:48.817084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:48.817120 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:48.818655 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.819028 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.819031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.819092 (+ 61us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.819191 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:48.819439 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:48.819442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:48.819444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:48.819651 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:48.819780 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:48.820016 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 16:06:48.820182 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260924 16:06:48.820735 14468 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51514
I20260924 16:06:48.821985 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14542
I20260924 16:06:48.830404 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14684
I20260924 16:06:48.839496 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14825
I20260924 16:06:48.845932 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 14437
2026-09-24T16:06:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2753 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[15041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[15041](info): set up 2 sockets
Sep 24 16:06:48 dist-test-slave-vj6n krb5kdc[15041](info): commencing operation
krb5kdc: starting...
W20260924 16:06:49.879222 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:49 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:50.001725 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35011
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:50.115278 15054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:50.115579 15054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:50.115661 15054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:50.119652 15054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:50.119733 15054 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 16:06:50.119766 15054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:50.119809 15054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:50.119843 15054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:50.119874 15054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:50.125036 15054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35011
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:50.126096 15054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:50.127149 15054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:50.135670 15060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.135667 15062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.135667 15059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:50.135900 15054 server_base.cc:1034] running on GCE node
I20260924 16:06:50.136520 15054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:50.137624 15054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:50.138839 15054 hybrid_clock.cc:648] HybridClock initialized: now 1790266010138828 us; error 33 us; skew 500 ppm
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:50.141660 15054 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:50.142618 15054 webserver.cc:460] Webserver started at http://127.13.8.126:35545/ using document root <none> and password file <none>
I20260924 16:06:50.143136 15054 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:50.143196 15054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:50.143360 15054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:50.145368 15054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "b6875892661d4b409744dfadac342844"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "9070078c93cb5b60d93aaf4729d2fd14"
server_key_iv: "896c84d65041767eec1d62b96955a338"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.145895 15054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "b6875892661d4b409744dfadac342844"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "9070078c93cb5b60d93aaf4729d2fd14"
server_key_iv: "896c84d65041767eec1d62b96955a338"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.149605 15054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:06:50.152083 15069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.153302 15054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:06:50.153437 15054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "b6875892661d4b409744dfadac342844"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "9070078c93cb5b60d93aaf4729d2fd14"
server_key_iv: "896c84d65041767eec1d62b96955a338"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.153542 15054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:50.205156 15054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:50.208181 15054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:50.208405 15054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:50.216470 15054 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35011
I20260924 16:06:50.216487 15132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35011 every 8 connection(s)
I20260924 16:06:50.217407 15054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:50.218886 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15054
I20260924 16:06:50.219009 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:50.219311 13345 external_mini_cluster.cc:1428] Setting key ba5a2da6b9e1714af310856d03f8d73e
I20260924 16:06:50.220718 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:50.226351 15133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Bootstrap starting.
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:50.228652 15133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:50.229477 15133 log.cc:824] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:50.231452 15133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: No bootstrap required, opened a new log
I20260924 16:06:50.233409 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.220683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35196 (local address 127.13.8.126:35011)
0924 16:06:50.221101 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.221116 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.221153 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:06:50.221780 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.221795 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.222109 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.222487 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.223276 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.224266 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.225106 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.225577 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.225737 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.228183 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.228187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.228200 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.228225 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.230571 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.231000 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.231005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.231010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.231135 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.231429 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.231433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.231435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.231889 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.232087 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.232369 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.232724 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":135,"threads_started":1}
I20260924 16:06:50.234503 15133 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } }
I20260924 16:06:50.234710 15133 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:50.234778 15133 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6875892661d4b409744dfadac342844, State: Initialized, Role: FOLLOWER
I20260924 16:06:50.235225 15133 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } }
I20260924 16:06:50.235370 15133 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:50.235440 15133 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:50.235548 15133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:50.236378 15133 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } }
I20260924 16:06:50.236704 15133 leader_election.cc:302] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6875892661d4b409744dfadac342844; no voters:
I20260924 16:06:50.236977 15133 leader_election.cc:288] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:50.237082 15138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:50.237339 15138 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [term 1 LEADER]: Becoming Leader. State: Replica: b6875892661d4b409744dfadac342844, State: Running, Role: LEADER
I20260924 16:06:50.237676 15138 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } }
I20260924 16:06:50.238127 15133 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:50.239326 15140 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6875892661d4b409744dfadac342844. Latest consensus state: current_term: 1 leader_uuid: "b6875892661d4b409744dfadac342844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } } }
I20260924 16:06:50.239500 15140 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:50.239871 15148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:50.240283 15139 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6875892661d4b409744dfadac342844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6875892661d4b409744dfadac342844" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35011 } } }
I20260924 16:06:50.240458 15139 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844 [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:50.242914 15148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:50.254755 15148 catalog_manager.cc:1347] Generated new cluster ID: d7cade0896ea4e789bc7afc126b94bab
I20260924 16:06:50.254828 15148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:50.261967 15148 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:50.262874 15148 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:50.266054 15148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Generated new TSK 0
I20260924 16:06:50.266706 15148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:06:50.296957 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35011
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:50.417440 15160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:50.417685 15160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:50.417728 15160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:50.421692 15160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:50.421742 15160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:50.421779 15160 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 16:06:50.421821 15160 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:50.421880 15160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:06:50.426860 15160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:35011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:50.427850 15160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:50.428860 15160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:50.438113 15165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.438149 15168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.438189 15166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:50.438288 15160 server_base.cc:1034] running on GCE node
I20260924 16:06:50.438840 15160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:50.439500 15160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:50.440677 15160 hybrid_clock.cc:648] HybridClock initialized: now 1790266010440637 us; error 50 us; skew 500 ppm
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:50.443328 15160 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:06:50.444115 15160 webserver.cc:460] Webserver started at http://127.13.8.65:34681/ using document root <none> and password file <none>
I20260924 16:06:50.444626 15160 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:50.444681 15160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:50.444832 15160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:50.446776 15160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "8be534cfd3614c669ef6c6757f3c87d3"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "58c6debb4f3a27c23900437d0ec58ee7"
server_key_iv: "8b5bc3cab14b7424d7f899823c58bf5f"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.447271 15160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "8be534cfd3614c669ef6c6757f3c87d3"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "58c6debb4f3a27c23900437d0ec58ee7"
server_key_iv: "8b5bc3cab14b7424d7f899823c58bf5f"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.450805 15160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:06:50.453279 15175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.454547 15160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:50.454663 15160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "8be534cfd3614c669ef6c6757f3c87d3"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "58c6debb4f3a27c23900437d0ec58ee7"
server_key_iv: "8b5bc3cab14b7424d7f899823c58bf5f"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.454748 15160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:50.474869 15160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:50.477968 15160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:50.478138 15160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:50.479833 15160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:50.479929 15160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.480001 15160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:50.480052 15160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:50.490792 15160 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:42561
I20260924 16:06:50.490849 15290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:42561 every 8 connection(s)
I20260924 16:06:50.491688 15160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:06:50.493428 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15160
I20260924 16:06:50.493564 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:06:50.493894 13345 external_mini_cluster.cc:1428] Setting key 72ecf49165100de8132a695724efa4cd
I20260924 16:06:50.495224 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.480513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:34761 (local address 127.13.8.126:35011)
0924 16:06:50.480685 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.480690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.482009 (+ 1319us) server_negotiation.cc:407] Connection header received
0924 16:06:50.483164 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.483169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.483233 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.483288 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.485106 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.486009 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.487378 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.487912 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.488070 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.491200 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.491203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.491206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.491235 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.492851 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.493481 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.493485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.493486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.493594 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.494029 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.494032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.494034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.494268 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.494364 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.494923 (+ 559us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.495050 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:06:50.495963 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.481383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.65:34761)
0924 16:06:50.481838 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.481888 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.482899 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.483679 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.483686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.484192 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.484913 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.484943 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.486182 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.486185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.487251 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.487261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.488130 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.488135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.488321 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.489011 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.489039 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.491023 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.492966 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.492972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.492996 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.493373 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.493686 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.493689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.493691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.493882 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.494375 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.494381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.494679 (+ 298us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.494858 (+ 179us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.495169 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":134,"threads_started":1}
I20260924 16:06:50.497972 15291 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35011
I20260924 16:06:50.498118 15291 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:50.498678 15291 heartbeater.cc:511] Master 127.13.8.126:35011 requested a full tablet report, sending...
I20260924 16:06:50.500273 15086 ts_manager.cc:194] Registered new tserver with Master: 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561)
I20260924 16:06:50.501958 15086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:34761
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:06:50.509305 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.500127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:33129 (local address 127.13.8.126:35011)
0924 16:06:50.500236 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.500240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.502466 (+ 2226us) server_negotiation.cc:407] Connection header received
0924 16:06:50.502580 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.502582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.502626 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.502701 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.503167 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.504033 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.504876 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.505300 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.505426 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.506493 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.506496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.506498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.506518 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.507738 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.508126 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.508128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.508130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.508196 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.508489 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.508495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.508497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.508726 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.508831 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.509113 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.509201 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:06:50.509513 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.500077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.65:33129)
0924 16:06:50.502388 (+ 2311us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.502399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.502505 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.502717 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.502720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.502927 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.503042 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.503049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.504188 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.504193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.504757 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.504765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.505414 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.505416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.505480 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.505988 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.505999 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.506359 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.507846 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.507849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.507851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.508047 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.508291 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.508293 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.508294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.508396 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.508834 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.508840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.508940 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.509057 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.509222 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:06:50.524868 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35011
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:50.633566 15300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:50.633863 15300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:50.633939 15300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:50.637952 15300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:50.638021 15300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:50.638088 15300 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 16:06:50.638126 15300 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:50.638188 15300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:06:50.643610 15300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:35011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:50.644688 15300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:50.645701 15300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:50.655776 15305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.655743 15306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.655753 15308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:50.656082 15300 server_base.cc:1034] running on GCE node
I20260924 16:06:50.656524 15300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:50.657189 15300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:50.658365 15300 hybrid_clock.cc:648] HybridClock initialized: now 1790266010658330 us; error 44 us; skew 500 ppm
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:50.661298 15300 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:06:50.662132 15300 webserver.cc:460] Webserver started at http://127.13.8.66:39579/ using document root <none> and password file <none>
I20260924 16:06:50.662720 15300 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:50.662781 15300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:50.662938 15300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:50.664848 15300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "3d6180f0f15c4f229da9c4363ef654af"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "fa9f1d2c60df8278e79a033025dfee68"
server_key_iv: "5e49bea3affbdc42b818d9b1af13b35a"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.665359 15300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "3d6180f0f15c4f229da9c4363ef654af"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "fa9f1d2c60df8278e79a033025dfee68"
server_key_iv: "5e49bea3affbdc42b818d9b1af13b35a"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.668936 15300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260924 16:06:50.671303 15315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.672499 15300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:06:50.672598 15300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "3d6180f0f15c4f229da9c4363ef654af"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "fa9f1d2c60df8278e79a033025dfee68"
server_key_iv: "5e49bea3affbdc42b818d9b1af13b35a"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.672683 15300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:50.693434 15300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:50.696727 15300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:50.696883 15300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:50.698441 15300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:50.698508 15300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.698580 15300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:50.698617 15300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:50.709869 15300 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38987
I20260924 16:06:50.710765 15433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38987 every 8 connection(s)
I20260924 16:06:50.711009 15300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:06:50.712815 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.699132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:46267 (local address 127.13.8.126:35011)
0924 16:06:50.699266 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.699270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.700339 (+ 1069us) server_negotiation.cc:407] Connection header received
0924 16:06:50.701321 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.701325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.701395 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.701477 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.702942 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.703873 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.705344 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.705672 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.705799 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.708784 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.708789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.708792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.708826 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.710540 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.711162 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.711165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.711167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.711271 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.711793 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.711796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.711798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.712013 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.712108 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.712600 (+ 492us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.712695 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 16:06:50.713609 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.699699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.66:46267)
0924 16:06:50.700198 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.700242 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.701099 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.701782 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.701793 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.702181 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.702786 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.702802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.704020 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.704026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.705187 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.705198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.705852 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.705857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.706042 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.706682 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.706704 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.708612 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.710651 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.710660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.710674 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.711030 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.711470 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.711474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.711476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.711648 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.712125 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.712130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.712405 (+ 275us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.712557 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.712786 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":143,"threads_started":1}
I20260924 16:06:50.714849 15434 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35011
I20260924 16:06:50.715000 15434 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:50.715487 15434 heartbeater.cc:511] Master 127.13.8.126:35011 requested a full tablet report, sending...
I20260924 16:06:50.716511 15085 ts_manager.cc:194] Registered new tserver with Master: 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987)
I20260924 16:06:50.717146 15085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:46267
I20260924 16:06:50.720891 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15300
I20260924 16:06:50.721016 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:06:50.721349 13345 external_mini_cluster.cc:1428] Setting key d0b537064af5a852cdb0291a0ff5c442
I20260924 16:06:50.723819 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.716935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:40567 (local address 127.13.8.126:35011)
0924 16:06:50.717058 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.717062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.717280 (+ 218us) server_negotiation.cc:407] Connection header received
0924 16:06:50.717416 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.717419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.717461 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.717537 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.717962 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.718528 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.719315 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.719528 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.719629 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.720601 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.720605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.720607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.720635 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.722131 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.722572 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.722578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.722580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.722663 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.722969 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.722972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.722972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.723159 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.723262 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.723597 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.723699 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:06:50.724042 15440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.716871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.66:40567)
0924 16:06:50.717188 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.717217 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.717319 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.717552 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.717555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.717747 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.717858 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.717863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.718642 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.718645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.719195 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.719202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.719635 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.719639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.719725 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.720145 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.720156 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.720473 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.722285 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.722287 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.722289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.722485 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.722762 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.722764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.722765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.722877 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.723287 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.723290 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.723405 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.723551 (+ 146us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.723740 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:06:50.748917 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35011
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:50.859683 15443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:50.859948 15443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:50.859992 15443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:50.863785 15443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:50.863834 15443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:50.863874 15443 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 16:06:50.863901 15443 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:06:50.863932 15443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:06:50.869076 15443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:35011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:50.870093 15443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:50.871135 15443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:50.880699 15448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.880719 15449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:50.880699 15451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:50.881126 15443 server_base.cc:1034] running on GCE node
I20260924 16:06:50.881547 15443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:50.882184 15443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:50.883415 15443 hybrid_clock.cc:648] HybridClock initialized: now 1790266010883401 us; error 31 us; skew 500 ppm
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:50.885965 15443 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:06:50.886852 15443 webserver.cc:460] Webserver started at http://127.13.8.67:45665/ using document root <none> and password file <none>
I20260924 16:06:50.887408 15443 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:50.887490 15443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:50.887694 15443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:50.889673 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "b34de93594d09885636f7fdef055176e"
server_key_iv: "417493b670881bb0697b3cb649f7df7e"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.890261 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "b34de93594d09885636f7fdef055176e"
server_key_iv: "417493b670881bb0697b3cb649f7df7e"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.893810 15443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:06:50.896185 15458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.897396 15443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:50.897526 15443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
format_stamp: "Formatted at 2026-09-24 16:06:50 on dist-test-slave-vj6n"
server_key: "b34de93594d09885636f7fdef055176e"
server_key_iv: "417493b670881bb0697b3cb649f7df7e"
server_key_version: "encryptionkey@0"
I20260924 16:06:50.897631 15443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:50.923250 15443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:50.926268 15443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:50.926433 15443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:50.927858 15443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:50.927906 15443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:50.927944 15443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:50.927995 15443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:50.939278 15443 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:45043
I20260924 16:06:50.939392 15574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:45043 every 8 connection(s)
I20260924 16:06:50.940394 15443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:06:50.942088 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.928803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:33239 (local address 127.13.8.126:35011)
0924 16:06:50.928950 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.928955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.929830 (+ 875us) server_negotiation.cc:407] Connection header received
0924 16:06:50.931135 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.931140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.931200 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.931306 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.932632 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.933419 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.934744 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.935032 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.935159 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.937815 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.937819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.937821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.937849 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.939658 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.940194 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.940198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.940200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.940294 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.940840 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.940844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.940846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.941130 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.941250 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.941860 (+ 610us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.941964 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:06:50.943013 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.929187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.67:33239)
0924 16:06:50.929663 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.929715 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.930910 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.931527 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.931535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.931941 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.932448 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.932480 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.933554 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.933558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.934605 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.934621 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.935211 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.935216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.935398 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.936026 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.936045 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.937582 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.939795 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.939800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.939812 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.940088 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.940400 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.940403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.940404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.940652 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.941310 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.941317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.941623 (+ 306us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.941798 (+ 175us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.942069 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":139,"threads_started":1}
I20260924 16:06:50.944267 15575 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35011
I20260924 16:06:50.944411 15575 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:50.944886 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15443
I20260924 16:06:50.945029 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:06:50.945021 15575 heartbeater.cc:511] Master 127.13.8.126:35011 requested a full tablet report, sending...
I20260924 16:06:50.945338 13345 external_mini_cluster.cc:1428] Setting key 9967c31fbefab2af494555f4da7f3d44
I20260924 16:06:50.946199 15085 ts_manager.cc:194] Registered new tserver with Master: 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:50.947031 15085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:33239
I20260924 16:06:50.949538 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:06:50.955286 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.946853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.13.8.67:54303)
0924 16:06:50.947447 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.947462 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.947708 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.948184 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.948187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.948485 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:50.948600 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.948609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.949493 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.949496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.950090 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.950099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.950597 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.950600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.950691 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.951214 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:50.951226 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:50.951550 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:50.953627 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.953630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.953631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.953843 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.954146 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:50.954149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:50.954150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:50.954296 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:50.954778 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:50.954780 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:50.954868 (+ 88us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.954965 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.955120 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":201,"threads_started":1}
I20260924 16:06:50.955365 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.946895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:54303 (local address 127.13.8.126:35011)
0924 16:06:50.947009 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.947013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.947623 (+ 610us) server_negotiation.cc:407] Connection header received
0924 16:06:50.947901 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.947905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.948015 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.948167 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.948780 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.949366 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.950210 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.950483 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.950591 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.951698 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.951703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.951709 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.951740 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.953510 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.953944 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.953951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.953956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.954048 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.954402 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.954409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.954413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.954674 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.954788 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.955059 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.955188 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 16:06:50 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:50.976959 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.968759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.13.8.126:35011)
0924 16:06:50.968899 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.968903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.968915 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:06:50.968957 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.968959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.968997 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.969099 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:50.969792 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.970400 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.971265 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.971488 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.971595 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.973913 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:50.973917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:50.973919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:50.973948 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:50.975446 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.975897 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.975901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.975903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.975976 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:50.976194 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:50.976196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:50.976197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:50.976374 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:50.976504 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.976737 (+ 233us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.976838 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 16:06:50.979947 15086 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:06:50.983031 15086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:50.997561 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.992641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.13.8.66:38987)
0924 16:06:50.993122 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.993133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.993160 (+ 27us) server_negotiation.cc:407] Connection header received
0924 16:06:50.993251 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.993255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.993424 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.993577 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:50.993802 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994869 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.996207 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.996868 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.996987 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.997153 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.997306 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.997396 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":102,"threads_started":1}
I20260924 16:06:50.997573 15595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.992202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38987 (local address 127.0.0.1:40386)
0924 16:06:50.993002 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.993019 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.993122 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.993560 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.993563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.993580 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:50.993680 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.993687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994989 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.994995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.996089 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.996098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.996986 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.996990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.997101 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.997140 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.997276 (+ 136us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.997396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":48,"thread_start_us":96,"threads_started":1}
I20260924 16:06:50.997908 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.991731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.13.8.65:42561)
0924 16:06:50.992127 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.992150 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.992598 (+ 448us) server_negotiation.cc:407] Connection header received
0924 16:06:50.992691 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.992697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.992925 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.993643 (+ 718us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:50.993784 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994591 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.996446 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.997236 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.997520 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.997661 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.997707 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.997786 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":80,"threads_started":1}
I20260924 16:06:50.997852 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.991492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.0.0.1:32986)
0924 16:06:50.991987 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.992036 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.992168 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.993130 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.993135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.993160 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:50.993346 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.993353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994700 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.994707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.996298 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.996309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.997398 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.997402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.997554 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.997576 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.997640 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.997709 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":202,"thread_start_us":72,"threads_started":1}
I20260924 16:06:50.999126 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.991681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45043 (local address 127.0.0.1:36664)
0924 16:06:50.992412 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:50.992427 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:50.992506 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:50.993269 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:50.993272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:50.993285 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:50.993362 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.993366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994791 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.994794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.996298 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:50.996309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.997382 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.997386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:50.997973 (+ 587us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.998065 (+ 92us) client_negotiation.cc:769] Sending connection context
0924 16:06:50.998126 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 16:06:50.999004 (+ 878us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":113,"thread_start_us":109,"threads_started":1}
I20260924 16:06:50.999207 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:50.992283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.13.8.67:45043)
0924 16:06:50.992774 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:50.992789 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:50.992812 (+ 23us) server_negotiation.cc:407] Connection header received
0924 16:06:50.992890 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:50.992895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:50.993093 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:50.993293 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:50.993431 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.994631 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.996446 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:50.997236 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:50.998576 (+ 1340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:50.998740 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:50.998947 (+ 207us) server_negotiation.cc:299] Negotiation successful
0924 16:06:50.999047 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":110,"threads_started":1}
I20260924 16:06:51.001197 15225 tablet_service.cc:1467] Processing CreateTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e (DEFAULT_TABLE table=test-table [id=463fc8509ed1453b989a577664ebc09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.001840 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e (DEFAULT_TABLE table=test-table [id=463fc8509ed1453b989a577664ebc09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.002506 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f73bc56f2ad498f97fc89827fdc759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.002903 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f73bc56f2ad498f97fc89827fdc759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.003922 15366 tablet_service.cc:1467] Processing CreateTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e (DEFAULT_TABLE table=test-table [id=463fc8509ed1453b989a577664ebc09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.004698 15366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f73bc56f2ad498f97fc89827fdc759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.010258 15599 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Bootstrap starting.
I20260924 16:06:51.010737 15598 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Bootstrap starting.
I20260924 16:06:51.012895 15598 tablet_bootstrap.cc:654] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.012998 15599 tablet_bootstrap.cc:654] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.013744 15599 log.cc:824] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:51.014056 15598 log.cc:824] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:51.014489 15600 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Bootstrap starting.
I20260924 16:06:51.015851 15599 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: No bootstrap required, opened a new log
I20260924 16:06:51.016052 15599 ts_tablet_manager.cc:1397] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260924 16:06:51.016106 15598 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: No bootstrap required, opened a new log
I20260924 16:06:51.016311 15598 ts_tablet_manager.cc:1397] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:06:51.016885 15600 tablet_bootstrap.cc:654] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.017707 15600 log.cc:824] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:51.019379 15599 raft_consensus.cc:348] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.019647 15599 raft_consensus.cc:374] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.019711 15599 raft_consensus.cc:729] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.019729 15600 tablet_bootstrap.cc:492] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: No bootstrap required, opened a new log
I20260924 16:06:51.019901 15600 ts_tablet_manager.cc:1397] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:06:51.020193 15599 consensus_queue.cc:260] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.020251 15598 raft_consensus.cc:348] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.020504 15598 raft_consensus.cc:374] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.020558 15598 raft_consensus.cc:729] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.020974 15598 consensus_queue.cc:260] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.021166 15599 ts_tablet_manager.cc:1428] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 16:06:51.021548 15575 heartbeater.cc:503] Master 127.13.8.126:35011 was elected leader, sending a full tablet report...
I20260924 16:06:51.021903 15598 ts_tablet_manager.cc:1428] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:06:51.022291 15291 heartbeater.cc:503] Master 127.13.8.126:35011 was elected leader, sending a full tablet report...
I20260924 16:06:51.023684 15600 raft_consensus.cc:348] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.023955 15600 raft_consensus.cc:374] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.024025 15600 raft_consensus.cc:729] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.024420 15600 consensus_queue.cc:260] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.025020 15434 heartbeater.cc:503] Master 127.13.8.126:35011 was elected leader, sending a full tablet report...
I20260924 16:06:51.025372 15600 ts_tablet_manager.cc:1428] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260924 16:06:51.174687 15604 raft_consensus.cc:482] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:51.174955 15604 raft_consensus.cc:504] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.176445 15604 leader_election.cc:288] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987)
I20260924 16:06:51.180398 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.176689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.13.8.67:46779)
0924 16:06:51.176884 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.176897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.177042 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.177464 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.177469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.177486 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.177590 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.177595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.178524 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.178529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.179297 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.179306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.179941 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.179944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.180084 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.180102 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.180186 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.180271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":24}
I20260924 16:06:51.180398 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.176730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46779 (local address 127.13.8.65:42561)
0924 16:06:51.176890 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.176896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.176968 (+ 72us) server_negotiation.cc:407] Connection header received
0924 16:06:51.177188 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.177191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.177265 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.177339 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.177692 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.178416 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.179416 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.179824 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.179940 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.180007 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.180204 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.180275 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:06:51.181111 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f73bc56f2ad498f97fc89827fdc759e" candidate_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3" is_pre_election: true
I20260924 16:06:51.181488 15245 raft_consensus.cc:2454] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f2ec30ac8af42ca859d1371a7cc7d05 in term 0.
I20260924 16:06:51.181604 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.176699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38987 (local address 127.13.8.67:46367)
0924 16:06:51.177320 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.177334 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.177424 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.177788 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.177792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.177812 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.178532 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.178544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.179381 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.179388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.180256 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.180265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.181148 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.181151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.181220 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.181249 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.181389 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.181455 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":177,"thread_start_us":172,"threads_started":1}
I20260924 16:06:51.181617 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.177035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46367 (local address 127.13.8.66:38987)
0924 16:06:51.177158 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.177163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.177419 (+ 256us) server_negotiation.cc:407] Connection header received
0924 16:06:51.177523 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.177526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.177596 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.177679 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.178704 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.179251 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.180384 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.181020 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.181147 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.181220 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.181410 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.181481 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 16:06:51.182053 15460 leader_election.cc:302] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8be534cfd3614c669ef6c6757f3c87d3, 8f2ec30ac8af42ca859d1371a7cc7d05; no voters:
I20260924 16:06:51.182372 15604 raft_consensus.cc:2781] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:51.182448 15604 raft_consensus.cc:482] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:51.182479 15604 raft_consensus.cc:3037] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.182420 15387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f73bc56f2ad498f97fc89827fdc759e" candidate_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6180f0f15c4f229da9c4363ef654af" is_pre_election: true
I20260924 16:06:51.182736 15387 raft_consensus.cc:2454] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f2ec30ac8af42ca859d1371a7cc7d05 in term 0.
I20260924 16:06:51.183630 15604 raft_consensus.cc:504] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.183959 15604 leader_election.cc:288] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [CANDIDATE]: Term 1 election: Requested vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987)
I20260924 16:06:51.184283 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f73bc56f2ad498f97fc89827fdc759e" candidate_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3"
I20260924 16:06:51.184338 15387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f73bc56f2ad498f97fc89827fdc759e" candidate_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6180f0f15c4f229da9c4363ef654af"
I20260924 16:06:51.184448 15387 raft_consensus.cc:3037] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.184424 15245 raft_consensus.cc:3037] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.185976 15387 raft_consensus.cc:2454] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f2ec30ac8af42ca859d1371a7cc7d05 in term 1.
I20260924 16:06:51.185976 15245 raft_consensus.cc:2454] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f2ec30ac8af42ca859d1371a7cc7d05 in term 1.
I20260924 16:06:51.186328 15460 leader_election.cc:302] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8be534cfd3614c669ef6c6757f3c87d3, 8f2ec30ac8af42ca859d1371a7cc7d05; no voters:
I20260924 16:06:51.186599 15604 raft_consensus.cc:2781] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:51.186785 15604 raft_consensus.cc:686] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 LEADER]: Becoming Leader. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Running, Role: LEADER
I20260924 16:06:51.187122 15604 consensus_queue.cc:237] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } }
I20260924 16:06:51.190661 15084 catalog_manager.cc:5495] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } health_report { overall_health: UNKNOWN } } }
W20260924 16:06:51.191514 15576 tablet.cc:2369] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:06:51.212280 15435 tablet.cc:2369] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:06:51.243917 15292 tablet.cc:2369] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 16:06:51 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:51.260275 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.252129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35011 (local address 127.0.0.1:35216)
0924 16:06:51.252504 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.252520 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.252695 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.252936 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.252938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.253329 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:51.253422 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.253428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.254103 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.254105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.254716 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.254724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.255111 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.255113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.255173 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.255755 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:51.255766 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:51.257158 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:51.258817 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:51.258826 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:51.258831 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:51.259081 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:51.259315 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:51.259317 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:51.259318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:51.259404 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:51.259773 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:51.259778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:51.259881 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.259974 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.260090 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260924 16:06:51.260275 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.252161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.13.8.126:35011)
0924 16:06:51.252307 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.252312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.252652 (+ 340us) server_negotiation.cc:407] Connection header received
0924 16:06:51.252793 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.252795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.252836 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.252896 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:51.253517 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.254015 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.254819 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.255022 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.255091 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.257283 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:51.257286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:51.257288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:51.257307 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:51.258717 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:51.259165 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:51.259167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:51.259169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:51.259241 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:51.259517 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:51.259520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:51.259521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:51.259682 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:51.259765 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.260027 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.260152 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":48}
I20260924 16:06:51.264344 15084 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:35216:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 16:06:51.265157 15084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:51.270184 15225 tablet_service.cc:1467] Processing CreateTablet for tablet 6c294e1b265645f8b92394ed40229481 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7657893a5bf7454e95014d3512666894]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:51.270146 15366 tablet_service.cc:1467] Processing CreateTablet for tablet 6c294e1b265645f8b92394ed40229481 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7657893a5bf7454e95014d3512666894]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:51.270171 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 6c294e1b265645f8b92394ed40229481 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7657893a5bf7454e95014d3512666894]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:06:51.270939 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c294e1b265645f8b92394ed40229481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.270937 15366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c294e1b265645f8b92394ed40229481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.270937 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c294e1b265645f8b92394ed40229481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.273389 15598 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: Bootstrap starting.
I20260924 16:06:51.273800 15600 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: Bootstrap starting.
I20260924 16:06:51.273800 15599 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: Bootstrap starting.
I20260924 16:06:51.274425 15598 tablet_bootstrap.cc:654] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.274737 15599 tablet_bootstrap.cc:654] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.274737 15600 tablet_bootstrap.cc:654] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.275888 15598 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: No bootstrap required, opened a new log
I20260924 16:06:51.275974 15598 ts_tablet_manager.cc:1397] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:06:51.276053 15599 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: No bootstrap required, opened a new log
I20260924 16:06:51.276136 15599 ts_tablet_manager.cc:1397] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:51.276163 15600 tablet_bootstrap.cc:492] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: No bootstrap required, opened a new log
I20260924 16:06:51.276233 15600 ts_tablet_manager.cc:1397] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:51.276505 15598 raft_consensus.cc:348] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.276629 15598 raft_consensus.cc:374] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.276672 15598 raft_consensus.cc:729] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.276654 15599 raft_consensus.cc:348] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.276788 15599 raft_consensus.cc:374] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.276834 15599 raft_consensus.cc:729] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.276815 15598 consensus_queue.cc:260] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.276957 15599 consensus_queue.cc:260] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.276998 15600 raft_consensus.cc:348] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.277098 15598 ts_tablet_manager.cc:1428] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:51.277107 15600 raft_consensus.cc:374] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.277140 15600 raft_consensus.cc:729] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.277143 15605 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.277241 15599 ts_tablet_manager.cc:1428] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:51.277274 15605 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.277259 15600 consensus_queue.cc:260] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.277438 15604 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.277523 15600 ts_tablet_manager.cc:1428] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260924 16:06:51.277575 15604 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.277740 15606 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.277881 15606 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.279325 15605 raft_consensus.cc:482] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:51.279414 15605 raft_consensus.cc:504] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.280478 15605 leader_election.cc:288] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:51.284190 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.280774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:58631 (local address 127.13.8.66:38987)
0924 16:06:51.280870 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.280875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.281092 (+ 217us) server_negotiation.cc:407] Connection header received
0924 16:06:51.281249 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.281252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.281292 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.281369 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.281612 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.282089 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.283131 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.283533 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.283636 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.283700 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.283997 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.284074 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 16:06:51.284567 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.280715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38987 (local address 127.13.8.65:58631)
0924 16:06:51.281005 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.281019 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.281143 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.281381 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.281384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.281400 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.281513 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.281518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.282189 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.282194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.283013 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.283022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.283669 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.283673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.283836 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.283860 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.284386 (+ 526us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.284454 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":39,"thread_start_us":121,"threads_started":1}
I20260924 16:06:51.284963 15387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c294e1b265645f8b92394ed40229481" candidate_uuid: "8be534cfd3614c669ef6c6757f3c87d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6180f0f15c4f229da9c4363ef654af" is_pre_election: true
I20260924 16:06:51.285096 15387 raft_consensus.cc:2454] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8be534cfd3614c669ef6c6757f3c87d3 in term 0.
I20260924 16:06:51.285122 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.281117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45043 (local address 127.13.8.65:38341)
0924 16:06:51.281434 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.281445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.281551 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.281782 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.281785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.281800 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.282266 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.282274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.282956 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.282961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.283848 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.283857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.284602 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.284605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.284801 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.284817 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.284929 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.284995 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":27,"thread_start_us":84,"threads_started":1}
I20260924 16:06:51.285131 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.281186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:38341 (local address 127.13.8.67:45043)
0924 16:06:51.281289 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.281293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.281514 (+ 221us) server_negotiation.cc:407] Connection header received
0924 16:06:51.281647 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.281650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.281693 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.281762 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.282381 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.282847 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.283966 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.284486 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.284601 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.284652 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.284957 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.285016 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 16:06:51.285610 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c294e1b265645f8b92394ed40229481" candidate_uuid: "8be534cfd3614c669ef6c6757f3c87d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" is_pre_election: true
I20260924 16:06:51.285681 15179 leader_election.cc:302] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:51.285820 15529 raft_consensus.cc:2454] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8be534cfd3614c669ef6c6757f3c87d3 in term 0.
I20260924 16:06:51.285972 15605 raft_consensus.cc:2781] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:51.286036 15605 raft_consensus.cc:482] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:51.286109 15605 raft_consensus.cc:3037] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.287127 15605 raft_consensus.cc:504] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.287448 15605 leader_election.cc:288] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [CANDIDATE]: Term 1 election: Requested vote from peers 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:51.287803 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c294e1b265645f8b92394ed40229481" candidate_uuid: "8be534cfd3614c669ef6c6757f3c87d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
I20260924 16:06:51.287827 15387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c294e1b265645f8b92394ed40229481" candidate_uuid: "8be534cfd3614c669ef6c6757f3c87d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6180f0f15c4f229da9c4363ef654af"
I20260924 16:06:51.287911 15529 raft_consensus.cc:3037] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.287946 15387 raft_consensus.cc:3037] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.288794 15387 raft_consensus.cc:2454] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8be534cfd3614c669ef6c6757f3c87d3 in term 1.
I20260924 16:06:51.288841 15529 raft_consensus.cc:2454] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8be534cfd3614c669ef6c6757f3c87d3 in term 1.
I20260924 16:06:51.289176 15179 leader_election.cc:302] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8be534cfd3614c669ef6c6757f3c87d3, 8f2ec30ac8af42ca859d1371a7cc7d05; no voters:
I20260924 16:06:51.289354 15605 raft_consensus.cc:2781] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:51.289561 15605 raft_consensus.cc:686] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 LEADER]: Becoming Leader. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Running, Role: LEADER
I20260924 16:06:51.289915 15605 consensus_queue.cc:237] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.292045 15621 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: TxnStatusTablet state changed. Reason: New leader 8be534cfd3614c669ef6c6757f3c87d3. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.292145 15621 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:51.292790 15623 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 16:06:51.293004 15623 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 16:06:51.293210 15084 catalog_manager.cc:5495] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65). New cstate: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260924 16:06:51.308033 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.304138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.0.0.1:32998)
0924 16:06:51.304277 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.304295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.304385 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.304726 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.304728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.304735 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:51.304824 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.304828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.305773 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.305775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.306419 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.306428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.307005 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.307008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.307068 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.307112 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.307807 (+ 695us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.307818 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.307865 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.307928 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 16:06:51.308053 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.304205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.13.8.65:42561)
0924 16:06:51.304318 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.304323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.304338 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:06:51.304517 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.304520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.304580 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.304712 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.304919 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.305668 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.306535 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.306892 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.306991 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.307243 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.307709 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.307826 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.307879 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.307932 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:06:51.312958 15387 raft_consensus.cc:1261] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Refusing update from remote peer 8be534cfd3614c669ef6c6757f3c87d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.312954 15529 raft_consensus.cc:1261] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Refusing update from remote peer 8be534cfd3614c669ef6c6757f3c87d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.313591 15605 consensus_queue.cc:1035] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.313781 15621 consensus_queue.cc:1035] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.318490 15606 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: TxnStatusTablet state changed. Reason: New leader 8be534cfd3614c669ef6c6757f3c87d3. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.318634 15606 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.318938 15604 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: TxnStatusTablet state changed. Reason: New leader 8be534cfd3614c669ef6c6757f3c87d3. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.319049 15604 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.320174 15621 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.320286 15621 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:51.320406 15626 mvcc.cc:205] Tried to move back new op lower bound from 7332929582330662912 to 7332929582246080512. Current Snapshot: MvccSnapshot[applied={T|T < 7332929582330662912}]
I20260924 16:06:51.320487 15628 mvcc.cc:205] Tried to move back new op lower bound from 7332929582330662912 to 7332929582246080512. Current Snapshot: MvccSnapshot[applied={T|T < 7332929582330662912}]
I20260924 16:06:51.320683 15606 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.320786 15606 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 3d6180f0f15c4f229da9c4363ef654af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.325407 15605 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.325513 15605 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8be534cfd3614c669ef6c6757f3c87d3: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:06:51.326665 15627 mvcc.cc:205] Tried to move back new op lower bound from 7332929582330662912 to 7332929582246080512. Current Snapshot: MvccSnapshot[applied={T|T < 7332929582330662912}]
I20260924 16:06:51.326799 15604 tablet_replica.cc:439] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8be534cfd3614c669ef6c6757f3c87d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } } }
I20260924 16:06:51.326915 15604 tablet_replica.cc:442] T 6c294e1b265645f8b92394ed40229481 P 8f2ec30ac8af42ca859d1371a7cc7d05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:06:51.333003 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.328812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.13.8.67:45043)
0924 16:06:51.328914 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.328920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.328959 (+ 39us) server_negotiation.cc:407] Connection header received
0924 16:06:51.329143 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.329146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.329202 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.329311 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.329557 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.330062 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.331290 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.331649 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.331753 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.332047 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.332619 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.332762 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.332822 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.332878 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:06:51.343662 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.340097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.13.8.67:41501)
0924 16:06:51.340284 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.340305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.340389 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.340840 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.340842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.340849 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:51.340952 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.340957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.341806 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.341810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.342492 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.342501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.342939 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.342942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.343040 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.343079 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.343379 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.343388 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.343463 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.343537 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 16:06:51.343662 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.340503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:41501 (local address 127.13.8.65:42561)
0924 16:06:51.340638 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.340642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.340652 (+ 10us) server_negotiation.cc:407] Connection header received
0924 16:06:51.340689 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.340691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.340733 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.340811 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.341081 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.341696 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.342601 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.342830 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.342927 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.343225 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.343298 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.343372 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.343485 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.343533 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 16:06:51.350106 15387 raft_consensus.cc:1261] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Refusing update from remote peer 8f2ec30ac8af42ca859d1371a7cc7d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.350349 15245 raft_consensus.cc:1261] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Refusing update from remote peer 8f2ec30ac8af42ca859d1371a7cc7d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.350605 15608 consensus_queue.cc:1035] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.350778 15604 consensus_queue.cc:1035] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.373817 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.369374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:41995 (local address 127.13.8.67:45043)
0924 16:06:51.369493 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.369498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.369678 (+ 180us) server_negotiation.cc:407] Connection header received
0924 16:06:51.369827 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.369831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.369899 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.369967 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.370564 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.371295 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.372587 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.372809 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.372880 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.373327 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.373397 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.373476 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.373622 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.373680 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 16:06:51.373829 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.369298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45043 (local address 127.13.8.65:41995)
0924 16:06:51.369603 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.369615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.369726 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.370084 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.370086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.370093 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:51.370184 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.370188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.371412 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.371416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.372460 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.372472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.372909 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.372913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.373009 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.373077 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.373505 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.373512 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.373581 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.373662 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260924 16:06:51.399899 15085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:51.400110 15085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:51.402868 15085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Sending DeleteTablet for 3 replicas of tablet 1f73bc56f2ad498f97fc89827fdc759e
I20260924 16:06:51.403569 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:51 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:36664
I20260924 16:06:51.404034 15225 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:51 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:32986
I20260924 16:06:51.404527 15647 tablet_replica.cc:330] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: stopping tablet replica
I20260924 16:06:51.404932 15647 raft_consensus.cc:2229] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:51.404899 15366 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f73bc56f2ad498f97fc89827fdc759e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:51 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:40386
I20260924 16:06:51.405192 15647 raft_consensus.cc:2258] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:51.406181 15084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:06:51.406637 15084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:51.406950 15647 ts_tablet_manager.cc:1905] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:51.409947 15647 ts_tablet_manager.cc:1918] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:51.410106 15647 log.cc:1196] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/1f73bc56f2ad498f97fc89827fdc759e
I20260924 16:06:51.410497 15647 ts_tablet_manager.cc:1939] T 1f73bc56f2ad498f97fc89827fdc759e P 8be534cfd3614c669ef6c6757f3c87d3: Deleting consensus metadata
I20260924 16:06:51.410758 15648 tablet_replica.cc:330] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: stopping tablet replica
I20260924 16:06:51.411972 15071 catalog_manager.cc:4841] TS 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561): tablet 1f73bc56f2ad498f97fc89827fdc759e (table test-table [id=463fc8509ed1453b989a577664ebc09d]) successfully deleted
I20260924 16:06:51.412168 15648 raft_consensus.cc:2229] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:51.412415 15648 raft_consensus.cc:2258] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:51.412825 15366 tablet_service.cc:1467] Processing CreateTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 (DEFAULT_TABLE table=test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.413125 15366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5a7b90a84d401c8e299fa7740b4173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.413921 15648 ts_tablet_manager.cc:1905] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:51.414485 15509 tablet_service.cc:1467] Processing CreateTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 (DEFAULT_TABLE table=test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.414780 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5a7b90a84d401c8e299fa7740b4173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.416762 15648 ts_tablet_manager.cc:1918] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:51.416898 15648 log.cc:1196] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/1f73bc56f2ad498f97fc89827fdc759e
I20260924 16:06:51.417249 15648 ts_tablet_manager.cc:1939] T 1f73bc56f2ad498f97fc89827fdc759e P 3d6180f0f15c4f229da9c4363ef654af: Deleting consensus metadata
I20260924 16:06:51.417433 15225 tablet_service.cc:1467] Processing CreateTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 (DEFAULT_TABLE table=test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:51.417717 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5a7b90a84d401c8e299fa7740b4173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:51.418586 15073 catalog_manager.cc:4841] TS 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987): tablet 1f73bc56f2ad498f97fc89827fdc759e (table test-table [id=463fc8509ed1453b989a577664ebc09d]) successfully deleted
I20260924 16:06:51.420023 15598 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Bootstrap starting.
I20260924 16:06:51.421330 15598 tablet_bootstrap.cc:654] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.421968 15600 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Bootstrap starting.
I20260924 16:06:51.422780 15598 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: No bootstrap required, opened a new log
I20260924 16:06:51.422856 15598 ts_tablet_manager.cc:1397] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:06:51.422895 15600 tablet_bootstrap.cc:654] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.422977 15599 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Bootstrap starting.
I20260924 16:06:51.423249 15598 raft_consensus.cc:348] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.423348 15598 raft_consensus.cc:374] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.423377 15598 raft_consensus.cc:729] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.423491 15598 consensus_queue.cc:260] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.423750 15598 ts_tablet_manager.cc:1428] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:51.423907 15599 tablet_bootstrap.cc:654] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:51.426430 15646 tablet_replica.cc:330] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: stopping tablet replica
I20260924 16:06:51.426820 15646 raft_consensus.cc:2229] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:06:51.426843 15599 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: No bootstrap required, opened a new log
I20260924 16:06:51.426988 15599 ts_tablet_manager.cc:1397] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 16:06:51.427434 15599 raft_consensus.cc:348] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.427560 15599 raft_consensus.cc:374] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.427588 15599 raft_consensus.cc:729] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.427702 15599 consensus_queue.cc:260] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.427948 15599 ts_tablet_manager.cc:1428] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:06:51.428853 15646 raft_consensus.cc:2258] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:51.430694 15646 ts_tablet_manager.cc:1905] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:51.431214 15600 tablet_bootstrap.cc:492] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: No bootstrap required, opened a new log
I20260924 16:06:51.431295 15600 ts_tablet_manager.cc:1397] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260924 16:06:51.431739 15600 raft_consensus.cc:348] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.431844 15600 raft_consensus.cc:374] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:51.431874 15600 raft_consensus.cc:729] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Initialized, Role: FOLLOWER
I20260924 16:06:51.431980 15600 consensus_queue.cc:260] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.432206 15600 ts_tablet_manager.cc:1428] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:51.435514 15646 ts_tablet_manager.cc:1918] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:51.435658 15646 log.cc:1196] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/1f73bc56f2ad498f97fc89827fdc759e
I20260924 16:06:51.436043 15646 ts_tablet_manager.cc:1939] T 1f73bc56f2ad498f97fc89827fdc759e P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting consensus metadata
I20260924 16:06:51.437258 15073 catalog_manager.cc:4841] TS 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043): tablet 1f73bc56f2ad498f97fc89827fdc759e (table test-table [id=463fc8509ed1453b989a577664ebc09d]) successfully deleted
I20260924 16:06:51.450712 15606 raft_consensus.cc:482] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:51.450860 15606 raft_consensus.cc:504] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.452273 15606 leader_election.cc:288] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:51.457513 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.453062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:37689 (local address 127.13.8.65:42561)
0924 16:06:51.453210 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.453214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.453227 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:51.453279 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.453282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.453325 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.453391 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.453775 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.454576 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.456160 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.456787 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.456862 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.456925 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.457303 (+ 378us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.457374 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:06:51.458107 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.452454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.13.8.66:37689)
0924 16:06:51.452910 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.452925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.453022 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.453507 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.453510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.453528 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.453643 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.453648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.454715 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.454719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.456020 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.456031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.457061 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.457064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.457164 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.457186 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.457806 (+ 620us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.457925 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":68,"threads_started":1}
I20260924 16:06:51.458685 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5a7b90a84d401c8e299fa7740b4173" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3" is_pre_election: true
I20260924 16:06:51.458828 15245 raft_consensus.cc:2454] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:51.459002 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.453246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:56373 (local address 127.13.8.67:45043)
0924 16:06:51.453370 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.453374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.454454 (+ 1080us) server_negotiation.cc:407] Connection header received
0924 16:06:51.454645 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.454649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.454698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.454747 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:06:51.455136 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.455872 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.457459 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.458037 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.458105 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.458181 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.458808 (+ 627us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.458891 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:06:51.459415 15317 leader_election.cc:302] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:51.459658 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.452607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45043 (local address 127.13.8.66:56373)
0924 16:06:51.454368 (+ 1761us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.454382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.454523 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.454879 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.454881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.454898 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:06:51.455006 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.455011 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.456006 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.456009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.457313 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.457325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.458397 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.458401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.458673 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.458696 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.459390 (+ 694us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.459462 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1695,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260924 16:06:51.459667 15606 raft_consensus.cc:2781] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:51.459996 15606 raft_consensus.cc:482] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:51.460047 15606 raft_consensus.cc:3037] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.460343 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5a7b90a84d401c8e299fa7740b4173" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" is_pre_election: true
I20260924 16:06:51.460500 15528 raft_consensus.cc:2454] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:51.461238 15606 raft_consensus.cc:504] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.461583 15606 leader_election.cc:288] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Requested vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:51.462447 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5a7b90a84d401c8e299fa7740b4173" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3"
I20260924 16:06:51.462551 15245 raft_consensus.cc:3037] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.462824 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5a7b90a84d401c8e299fa7740b4173" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
I20260924 16:06:51.462956 15528 raft_consensus.cc:3037] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:51.463662 15245 raft_consensus.cc:2454] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:51.463973 15317 leader_election.cc:302] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:51.464079 15528 raft_consensus.cc:2454] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:51.464119 15606 raft_consensus.cc:2781] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:51.464287 15606 raft_consensus.cc:686] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Becoming Leader. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Running, Role: LEADER
I20260924 16:06:51.464519 15606 consensus_queue.cc:237] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:51.466828 15085 catalog_manager.cc:5495] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66). New cstate: current_term: 1 leader_uuid: "3d6180f0f15c4f229da9c4363ef654af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260924 16:06:51.488545 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.484843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.13.8.66:38987)
0924 16:06:51.484966 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.484971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.484984 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:51.485057 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.485059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.485117 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.485221 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.485460 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.486234 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.487017 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.487347 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.487446 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.487657 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.488213 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.488325 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.488363 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.488412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 16:06:51.502302 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.496331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:34287 (local address 127.13.8.65:42561)
0924 16:06:51.496439 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.496444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.498300 (+ 1856us) server_negotiation.cc:407] Connection header received
0924 16:06:51.498446 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.498449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.498503 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.498591 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.498972 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.499770 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.500955 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.501301 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.501395 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.501774 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.501864 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.501934 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.502081 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.502132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:06:51.502347 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.496232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.13.8.66:34287)
0924 16:06:51.498195 (+ 1963us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.498211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.498344 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.498719 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.498722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.498746 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:51.498858 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.498864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.499877 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.499880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.500844 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.500855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.501479 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.501482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.501575 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.501624 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.501989 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.501996 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.502060 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.502140 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1842,"thread_start_us":104,"threads_started":1}
I20260924 16:06:51.509838 15245 raft_consensus.cc:1261] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.510407 15528 raft_consensus.cc:1261] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:51.510383 15654 consensus_queue.cc:1035] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.510798 15654 consensus_queue.cc:1035] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:51.529011 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.525081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38987 (local address 127.13.8.65:34333)
0924 16:06:51.525206 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:51.525223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:51.525321 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:51.525831 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:51.525834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:51.525843 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:06:51.525936 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.525942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.526932 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.526936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.527734 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:51.527744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.528334 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.528338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:51.528410 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.528440 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.528741 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.528747 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:06:51.528801 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 16:06:51.528885 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 16:06:51.529609 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:51.525455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:34333 (local address 127.13.8.66:38987)
0924 16:06:51.525547 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:51.525552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:51.525564 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:06:51.525604 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:51.525607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:51.525660 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:51.525728 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:51.526037 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.526816 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.527870 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:51.528163 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:51.528253 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:51.528562 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:51.528639 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:51.528706 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:51.529446 (+ 740us) server_negotiation.cc:299] Negotiation successful
0924 16:06:51.529499 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260924 16:06:51.530176 15628 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 16:06:53.094806 15085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:53.094981 15085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:53.097177 15085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Sending DeleteTablet for 3 replicas of tablet dc5a7b90a84d401c8e299fa7740b4173
I20260924 16:06:53.097800 15366 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:40386
I20260924 16:06:53.097880 15225 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:32986
I20260924 16:06:53.098008 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5a7b90a84d401c8e299fa7740b4173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:36664
I20260924 16:06:53.098187 15686 tablet_replica.cc:330] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: stopping tablet replica
I20260924 16:06:53.098385 15687 tablet_replica.cc:330] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: stopping tablet replica
I20260924 16:06:53.098474 15688 tablet_replica.cc:330] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: stopping tablet replica
I20260924 16:06:53.098488 15686 raft_consensus.cc:2229] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:06:53.098538 15687 raft_consensus.cc:2229] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:53.098790 15687 raft_consensus.cc:2258] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.099542 15687 ts_tablet_manager.cc:1905] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:53.099695 15686 raft_consensus.cc:2258] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.100283 15688 raft_consensus.cc:2229] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:53.100405 15688 raft_consensus.cc:2258] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.100597 15686 ts_tablet_manager.cc:1905] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:53.100950 15688 ts_tablet_manager.cc:1905] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:53.102546 15687 ts_tablet_manager.cc:1918] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 16:06:53.102633 15687 log.cc:1196] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/dc5a7b90a84d401c8e299fa7740b4173
I20260924 16:06:53.102913 15687 ts_tablet_manager.cc:1939] T dc5a7b90a84d401c8e299fa7740b4173 P 8be534cfd3614c669ef6c6757f3c87d3: Deleting consensus metadata
I20260924 16:06:53.103578 15071 catalog_manager.cc:4841] TS 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561): tablet dc5a7b90a84d401c8e299fa7740b4173 (table test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]) successfully deleted
I20260924 16:06:53.103709 15688 ts_tablet_manager.cc:1918] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 16:06:53.103802 15688 log.cc:1196] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/dc5a7b90a84d401c8e299fa7740b4173
I20260924 16:06:53.104058 15688 ts_tablet_manager.cc:1939] T dc5a7b90a84d401c8e299fa7740b4173 P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting consensus metadata
I20260924 16:06:53.104545 15686 ts_tablet_manager.cc:1918] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 16:06:53.104650 15686 log.cc:1196] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/dc5a7b90a84d401c8e299fa7740b4173
I20260924 16:06:53.104800 15073 catalog_manager.cc:4841] TS 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043): tablet dc5a7b90a84d401c8e299fa7740b4173 (table test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]) successfully deleted
I20260924 16:06:53.105075 15686 ts_tablet_manager.cc:1939] T dc5a7b90a84d401c8e299fa7740b4173 P 3d6180f0f15c4f229da9c4363ef654af: Deleting consensus metadata
I20260924 16:06:53.105770 15073 catalog_manager.cc:4841] TS 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987): tablet dc5a7b90a84d401c8e299fa7740b4173 (table test-table [id=39e68f9e1f184dd0a6e48da800a58e2f]) successfully deleted
Sep 24 16:06:53 dist-test-slave-vj6n krb5kdc[15041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 16:06:53 dist-test-slave-vj6n krb5kdc[15041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:53.130986 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:53.118056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.13.8.126:35011)
0924 16:06:53.118311 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:53.118318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:53.118331 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:06:53.118431 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:53.118433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:53.118472 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:53.118548 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:53.119189 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:53.119713 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:53.120517 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:53.120741 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:53.120812 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:53.128010 (+ 7198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:53.128017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:53.128022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:53.128059 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:53.129425 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:53.129814 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:53.129817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:53.129818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:53.129889 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:53.130108 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:53.130110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:53.130111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:53.130397 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:53.130500 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:53.130749 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 16:06:53.130855 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
I20260924 16:06:53.133586 15085 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:06:53.134044 15085 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:53.138768 15225 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f (DEFAULT_TABLE table=test-table [id=fce2c911b4014df88f64093bf1ecd0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.138823 15366 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f (DEFAULT_TABLE table=test-table [id=fce2c911b4014df88f64093bf1ecd0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.138834 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f (DEFAULT_TABLE table=test-table [id=fce2c911b4014df88f64093bf1ecd0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.139084 15366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff8e293ca3f4bc590d0dc600769c44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.139083 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff8e293ca3f4bc590d0dc600769c44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.139166 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ff8e293ca3f4bc590d0dc600769c44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.141309 15697 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Bootstrap starting.
I20260924 16:06:53.141628 15698 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Bootstrap starting.
I20260924 16:06:53.141911 15699 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Bootstrap starting.
I20260924 16:06:53.142513 15698 tablet_bootstrap.cc:654] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.142674 15697 tablet_bootstrap.cc:654] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.142807 15699 tablet_bootstrap.cc:654] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.143625 15698 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: No bootstrap required, opened a new log
I20260924 16:06:53.143724 15698 ts_tablet_manager.cc:1397] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 16:06:53.143971 15699 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: No bootstrap required, opened a new log
I20260924 16:06:53.144085 15699 ts_tablet_manager.cc:1397] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:53.144176 15698 raft_consensus.cc:348] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.144316 15698 raft_consensus.cc:374] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.144340 15698 raft_consensus.cc:729] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.144464 15698 consensus_queue.cc:260] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.144526 15699 raft_consensus.cc:348] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.144644 15699 raft_consensus.cc:374] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.144690 15699 raft_consensus.cc:729] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.144766 15698 ts_tablet_manager.cc:1428] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:53.144894 15699 consensus_queue.cc:260] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.145198 15699 ts_tablet_manager.cc:1428] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:53.146070 15697 tablet_bootstrap.cc:492] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: No bootstrap required, opened a new log
I20260924 16:06:53.146190 15697 ts_tablet_manager.cc:1397] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260924 16:06:53.146687 15697 raft_consensus.cc:348] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.146816 15697 raft_consensus.cc:374] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.146849 15697 raft_consensus.cc:729] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.146971 15697 consensus_queue.cc:260] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.147357 15697 ts_tablet_manager.cc:1428] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 16:06:53.163147 15681 raft_consensus.cc:482] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:53.163298 15681 raft_consensus.cc:504] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.163784 15681 leader_election.cc:288] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:53.164284 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff8e293ca3f4bc590d0dc600769c44f" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" is_pre_election: true
I20260924 16:06:53.164278 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff8e293ca3f4bc590d0dc600769c44f" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3" is_pre_election: true
I20260924 16:06:53.164443 15528 raft_consensus.cc:2454] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:53.164444 15245 raft_consensus.cc:2454] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:53.164798 15317 leader_election.cc:302] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:53.164971 15681 raft_consensus.cc:2781] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:53.165038 15681 raft_consensus.cc:482] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:53.165095 15681 raft_consensus.cc:3037] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.166167 15681 raft_consensus.cc:504] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.166649 15681 leader_election.cc:288] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Requested vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:53.166993 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff8e293ca3f4bc590d0dc600769c44f" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3"
I20260924 16:06:53.167047 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff8e293ca3f4bc590d0dc600769c44f" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
I20260924 16:06:53.167140 15245 raft_consensus.cc:3037] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.167160 15528 raft_consensus.cc:3037] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.168311 15528 raft_consensus.cc:2454] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:53.168311 15245 raft_consensus.cc:2454] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:53.168637 15317 leader_election.cc:302] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:53.168828 15681 raft_consensus.cc:2781] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:53.168972 15681 raft_consensus.cc:686] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Becoming Leader. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Running, Role: LEADER
I20260924 16:06:53.169148 15681 consensus_queue.cc:237] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.171002 15085 catalog_manager.cc:5495] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66). New cstate: current_term: 1 leader_uuid: "3d6180f0f15c4f229da9c4363ef654af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260924 16:06:53.192782 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:53.189367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.13.8.66:38987)
0924 16:06:53.189673 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:53.189680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:53.189692 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:06:53.189730 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:53.189732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:53.189772 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:53.189842 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:06:53.190177 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:53.190875 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:53.191795 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:53.192013 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:53.192102 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:53.192327 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:06:53.192395 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:06:53.192448 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:53.192527 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 16:06:53.192578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":113,"threads_started":1}
I20260924 16:06:53.198644 15245 raft_consensus.cc:1261] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:53.198694 15528 raft_consensus.cc:1261] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:53.199090 15661 consensus_queue.cc:1035] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:53.199234 15681 consensus_queue.cc:1035] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:53.232676 15084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:53.232812 15084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:53.234665 15084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Sending DeleteTablet for 3 replicas of tablet 0ff8e293ca3f4bc590d0dc600769c44f
I20260924 16:06:53.235136 15225 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:32986
I20260924 16:06:53.235222 15366 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:40386
I20260924 16:06:53.235224 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff8e293ca3f4bc590d0dc600769c44f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:53 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:36664
I20260924 16:06:53.235391 15688 tablet_replica.cc:330] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: stopping tablet replica
I20260924 16:06:53.235397 15687 tablet_replica.cc:330] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: stopping tablet replica
I20260924 16:06:53.235512 15687 raft_consensus.cc:2229] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:53.235512 15688 raft_consensus.cc:2229] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:53.235527 15686 tablet_replica.cc:330] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: stopping tablet replica
I20260924 16:06:53.235621 15688 raft_consensus.cc:2258] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.235621 15687 raft_consensus.cc:2258] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.235693 15686 raft_consensus.cc:2229] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:06:53.235884 15686 raft_consensus.cc:2258] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:53.236344 15688 ts_tablet_manager.cc:1905] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:53.236402 15687 ts_tablet_manager.cc:1905] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:53.236378 15084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 16:06:53.236596 15686 ts_tablet_manager.cc:1905] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 16:06:53.236754 15084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:06:53.239068 15688 ts_tablet_manager.cc:1918] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:53.239073 15687 ts_tablet_manager.cc:1918] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:53.239141 15688 log.cc:1196] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/0ff8e293ca3f4bc590d0dc600769c44f
I20260924 16:06:53.239140 15686 ts_tablet_manager.cc:1918] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:06:53.239213 15687 log.cc:1196] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/0ff8e293ca3f4bc590d0dc600769c44f
I20260924 16:06:53.239200 15686 log.cc:1196] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/0ff8e293ca3f4bc590d0dc600769c44f
I20260924 16:06:53.239420 15688 ts_tablet_manager.cc:1939] T 0ff8e293ca3f4bc590d0dc600769c44f P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting consensus metadata
I20260924 16:06:53.239537 15687 ts_tablet_manager.cc:1939] T 0ff8e293ca3f4bc590d0dc600769c44f P 8be534cfd3614c669ef6c6757f3c87d3: Deleting consensus metadata
I20260924 16:06:53.239547 15686 ts_tablet_manager.cc:1939] T 0ff8e293ca3f4bc590d0dc600769c44f P 3d6180f0f15c4f229da9c4363ef654af: Deleting consensus metadata
I20260924 16:06:53.240458 15073 catalog_manager.cc:4841] TS 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987): tablet 0ff8e293ca3f4bc590d0dc600769c44f (table test-table [id=fce2c911b4014df88f64093bf1ecd0c0]) successfully deleted
I20260924 16:06:53.240597 15073 catalog_manager.cc:4841] TS 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043): tablet 0ff8e293ca3f4bc590d0dc600769c44f (table test-table [id=fce2c911b4014df88f64093bf1ecd0c0]) successfully deleted
I20260924 16:06:53.241184 15071 catalog_manager.cc:4841] TS 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561): tablet 0ff8e293ca3f4bc590d0dc600769c44f (table test-table [id=fce2c911b4014df88f64093bf1ecd0c0]) successfully deleted
I20260924 16:06:53.242172 15225 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1776533dd43c0a836d55abc094dfa (DEFAULT_TABLE table=test-table [id=d660fb559f5a4e1cbfac8c92788110c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.242275 15366 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1776533dd43c0a836d55abc094dfa (DEFAULT_TABLE table=test-table [id=d660fb559f5a4e1cbfac8c92788110c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.242355 15509 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1776533dd43c0a836d55abc094dfa (DEFAULT_TABLE table=test-table [id=d660fb559f5a4e1cbfac8c92788110c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:06:53.242444 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1776533dd43c0a836d55abc094dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.242527 15366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1776533dd43c0a836d55abc094dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.242692 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1776533dd43c0a836d55abc094dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:53.244240 15699 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Bootstrap starting.
I20260924 16:06:53.244814 15698 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3: Bootstrap starting.
I20260924 16:06:53.244834 15697 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Bootstrap starting.
I20260924 16:06:53.245083 15699 tablet_bootstrap.cc:654] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.245630 15697 tablet_bootstrap.cc:654] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.245779 15698 tablet_bootstrap.cc:654] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:53.246268 15699 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: No bootstrap required, opened a new log
I20260924 16:06:53.246361 15699 ts_tablet_manager.cc:1397] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:53.246784 15697 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: No bootstrap required, opened a new log
I20260924 16:06:53.246884 15697 ts_tablet_manager.cc:1397] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260924 16:06:53.246824 15699 raft_consensus.cc:348] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.246915 15698 tablet_bootstrap.cc:492] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3: No bootstrap required, opened a new log
I20260924 16:06:53.246930 15699 raft_consensus.cc:374] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.246968 15699 raft_consensus.cc:729] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.246999 15698 ts_tablet_manager.cc:1397] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260924 16:06:53.247102 15699 consensus_queue.cc:260] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.247287 15697 raft_consensus.cc:348] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.247367 15699 ts_tablet_manager.cc:1428] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:06:53.247401 15697 raft_consensus.cc:374] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.247445 15697 raft_consensus.cc:729] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2ec30ac8af42ca859d1371a7cc7d05, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.247426 15698 raft_consensus.cc:348] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.247534 15698 raft_consensus.cc:374] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:53.247568 15698 raft_consensus.cc:729] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be534cfd3614c669ef6c6757f3c87d3, State: Initialized, Role: FOLLOWER
I20260924 16:06:53.247560 15697 consensus_queue.cc:260] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.247692 15698 consensus_queue.cc:260] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.247855 15697 ts_tablet_manager.cc:1428] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260924 16:06:53.247954 15698 ts_tablet_manager.cc:1428] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 16:06:53.276886 15679 raft_consensus.cc:482] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:06:53.277014 15679 raft_consensus.cc:504] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.277480 15679 leader_election.cc:288] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:53.277913 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1776533dd43c0a836d55abc094dfa" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3" is_pre_election: true
I20260924 16:06:53.278056 15245 raft_consensus.cc:2454] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:53.277987 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1776533dd43c0a836d55abc094dfa" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" is_pre_election: true
I20260924 16:06:53.278193 15528 raft_consensus.cc:2454] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 0.
I20260924 16:06:53.278407 15317 leader_election.cc:302] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3; no voters:
I20260924 16:06:53.278577 15679 raft_consensus.cc:2781] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:06:53.278635 15679 raft_consensus.cc:482] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:06:53.278664 15679 raft_consensus.cc:3037] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.279676 15679 raft_consensus.cc:504] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.280057 15679 leader_election.cc:288] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Requested vote from peers 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561), 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043)
I20260924 16:06:53.280386 15245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1776533dd43c0a836d55abc094dfa" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be534cfd3614c669ef6c6757f3c87d3"
I20260924 16:06:53.280550 15245 raft_consensus.cc:3037] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.280526 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1776533dd43c0a836d55abc094dfa" candidate_uuid: "3d6180f0f15c4f229da9c4363ef654af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05"
I20260924 16:06:53.280681 15528 raft_consensus.cc:3037] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:53.281831 15245 raft_consensus.cc:2454] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:53.281829 15528 raft_consensus.cc:2454] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d6180f0f15c4f229da9c4363ef654af in term 1.
I20260924 16:06:53.282155 15317 leader_election.cc:302] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d6180f0f15c4f229da9c4363ef654af, 8be534cfd3614c669ef6c6757f3c87d3, 8f2ec30ac8af42ca859d1371a7cc7d05; no voters:
I20260924 16:06:53.282349 15679 raft_consensus.cc:2781] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:53.282454 15679 raft_consensus.cc:686] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Becoming Leader. State: Replica: 3d6180f0f15c4f229da9c4363ef654af, State: Running, Role: LEADER
I20260924 16:06:53.282615 15679 consensus_queue.cc:237] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } }
I20260924 16:06:53.284211 15086 catalog_manager.cc:5495] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66). New cstate: current_term: 1 leader_uuid: "3d6180f0f15c4f229da9c4363ef654af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d6180f0f15c4f229da9c4363ef654af" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260924 16:06:53.310683 15528 raft_consensus.cc:1261] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:53.310711 15245 raft_consensus.cc:1261] T e1f1776533dd43c0a836d55abc094dfa P 8be534cfd3614c669ef6c6757f3c87d3 [term 1 FOLLOWER]: Refusing update from remote peer 3d6180f0f15c4f229da9c4363ef654af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:06:53.311064 15661 consensus_queue.cc:1035] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f2ec30ac8af42ca859d1371a7cc7d05" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:06:53.311247 15679 consensus_queue.cc:1035] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be534cfd3614c669ef6c6757f3c87d3" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 16:06:53.325112 15669 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 16:06:54.876590 15084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:54.876732 15084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:06:54.879072 15084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6875892661d4b409744dfadac342844: Sending DeleteTablet for 3 replicas of tablet e1f1776533dd43c0a836d55abc094dfa
I20260924 16:06:54.879606 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1776533dd43c0a836d55abc094dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:54 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:32986
I20260924 16:06:54.879698 15366 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1776533dd43c0a836d55abc094dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:54 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:40386
I20260924 16:06:54.879750 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1776533dd43c0a836d55abc094dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:06:54 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:36664
I20260924 16:06:54.879985 15739 tablet_replica.cc:330] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: stopping tablet replica
I20260924 16:06:54.880070 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15160
I20260924 16:06:54.880138 15740 tablet_replica.cc:330] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: stopping tablet replica
I20260924 16:06:54.880290 15740 raft_consensus.cc:2229] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:06:54.880393 15740 raft_consensus.cc:2258] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:54.880119 15739 raft_consensus.cc:2229] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:06:54.880980 15739 raft_consensus.cc:2258] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:06:54.881997 15740 ts_tablet_manager.cc:1905] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:54.882357 15739 ts_tablet_manager.cc:1905] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:06:54.884859 15740 ts_tablet_manager.cc:1918] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 16:06:54.884939 15740 log.cc:1196] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/e1f1776533dd43c0a836d55abc094dfa
I20260924 16:06:54.884948 15739 ts_tablet_manager.cc:1918] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 16:06:54.885010 15739 log.cc:1196] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/e1f1776533dd43c0a836d55abc094dfa
I20260924 16:06:54.885218 15740 ts_tablet_manager.cc:1939] T e1f1776533dd43c0a836d55abc094dfa P 8f2ec30ac8af42ca859d1371a7cc7d05: Deleting consensus metadata
I20260924 16:06:54.885293 15739 ts_tablet_manager.cc:1939] T e1f1776533dd43c0a836d55abc094dfa P 3d6180f0f15c4f229da9c4363ef654af: Deleting consensus metadata
I20260924 16:06:54.885917 15073 catalog_manager.cc:4841] TS 3d6180f0f15c4f229da9c4363ef654af (127.13.8.66:38987): tablet e1f1776533dd43c0a836d55abc094dfa (table test-table [id=d660fb559f5a4e1cbfac8c92788110c6]) successfully deleted
I20260924 16:06:54.886109 15073 catalog_manager.cc:4841] TS 8f2ec30ac8af42ca859d1371a7cc7d05 (127.13.8.67:45043): tablet e1f1776533dd43c0a836d55abc094dfa (table test-table [id=d660fb559f5a4e1cbfac8c92788110c6]) successfully deleted
W20260924 16:06:54.888419 15071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:42561 (error 108)
I20260924 16:06:54.888643 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15300
I20260924 16:06:54.890534 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:54.889762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42561 (local address 127.0.0.1:33004)
0924 16:06:54.890292 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:54.890380 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:42561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":64,"threads_started":1}
W20260924 16:06:54.890650 15071 catalog_manager.cc:4584] TS 8be534cfd3614c669ef6c6757f3c87d3 (127.13.8.65:42561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1f1776533dd43c0a836d55abc094dfa: Network error: Client connection negotiation failed: client connection to 127.13.8.65:42561: connect: Connection refused (error 111)
I20260924 16:06:54.896414 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15443
I20260924 16:06:54.903115 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15054
2026-09-24T16:06:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6058 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:54 dist-test-slave-vj6n krb5kdc[15745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:54 dist-test-slave-vj6n krb5kdc[15745](info): set up 2 sockets
Sep 24 16:06:54 dist-test-slave-vj6n krb5kdc[15745](info): commencing operation
krb5kdc: starting...
W20260924 16:06:55.969774 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:56 dist-test-slave-vj6n krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:56.146610 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44173
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:44173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:56.353839 15758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:56.354403 15758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:56.354585 15758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:56.360400 15758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:56.360551 15758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:56.360632 15758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:56.360731 15758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:56.360823 15758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:56.368778 15758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:44173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:56.370704 15758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:56.372483 15758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:56.385939 15766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:56.386042 15763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:56.386075 15764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:56.386875 15758 server_base.cc:1034] running on GCE node
I20260924 16:06:56.387740 15758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:56.389518 15758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:56.390851 15758 hybrid_clock.cc:648] HybridClock initialized: now 1790266016390804 us; error 61 us; skew 500 ppm
Sep 24 16:06:56 dist-test-slave-vj6n krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:56.396075 15758 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:56.397604 15758 webserver.cc:460] Webserver started at http://127.13.8.126:42079/ using document root <none> and password file <none>
I20260924 16:06:56.398543 15758 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:56.398648 15758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:56.398923 15758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:56.402200 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "429605ee3615451fa15b3516db2ab51e"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "3b7bff70b63a46585b8966080c7a9ec4"
server_key_iv: "f17597ca42fc6c26440987f2fdf385bd"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.403198 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "429605ee3615451fa15b3516db2ab51e"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "3b7bff70b63a46585b8966080c7a9ec4"
server_key_iv: "f17597ca42fc6c26440987f2fdf385bd"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.409579 15758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260924 16:06:56.413661 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:56.415741 15758 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:06:56.415896 15758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "429605ee3615451fa15b3516db2ab51e"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "3b7bff70b63a46585b8966080c7a9ec4"
server_key_iv: "f17597ca42fc6c26440987f2fdf385bd"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.416034 15758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:56.431896 15758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:56.436486 15758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:56.436719 15758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:56.448596 15758 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:44173
I20260924 16:06:56.448640 15826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:44173 every 8 connection(s)
I20260924 16:06:56.450192 15758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:56.455103 15827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:56.457984 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15758
I20260924 16:06:56.458106 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:56.458559 13345 external_mini_cluster.cc:1428] Setting key 1151d55a9c106c7271a34c222650b4ee
I20260924 16:06:56.465451 15827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e: Bootstrap starting.
I20260924 16:06:56.469182 15827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:56.470431 15827 log.cc:824] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:06:56 dist-test-slave-vj6n krb5kdc[15745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:56.473796 15827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e: No bootstrap required, opened a new log
I20260924 16:06:56.481173 15827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } }
I20260924 16:06:56.481597 15827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:56.481698 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:56.461048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.13.8.126:44173)
0924 16:06:56.461805 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:56.461839 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:56.461901 (+ 62us) server_negotiation.cc:407] Connection header received
0924 16:06:56.462984 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:56.463018 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:56.463561 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:56.464135 (+ 574us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:56.465593 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.467212 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.468563 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.469425 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.469712 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:56.474067 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:56.474082 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:56.474105 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:56.474141 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:56.476941 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:56.477703 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:56.477723 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:56.477730 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:56.477902 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:56.478480 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:56.478487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:56.478488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:56.479177 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:56.479448 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:56.479926 (+ 478us) server_negotiation.cc:299] Negotiation successful
0924 16:06:56.480410 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":245,"threads_started":1}
I20260924 16:06:56.481734 15827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429605ee3615451fa15b3516db2ab51e, State: Initialized, Role: FOLLOWER
I20260924 16:06:56.482906 15827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } }
I20260924 16:06:56.483168 15827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:56.483289 15827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:56.483491 15827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:56.484910 15827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } }
I20260924 16:06:56.485522 15827 leader_election.cc:302] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 429605ee3615451fa15b3516db2ab51e; no voters:
I20260924 16:06:56.486059 15827 leader_election.cc:288] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:56.486265 15832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:56.486613 15832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [term 1 LEADER]: Becoming Leader. State: Replica: 429605ee3615451fa15b3516db2ab51e, State: Running, Role: LEADER
I20260924 16:06:56.487217 15832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } }
I20260924 16:06:56.488174 15827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:56.489358 15834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 429605ee3615451fa15b3516db2ab51e. Latest consensus state: current_term: 1 leader_uuid: "429605ee3615451fa15b3516db2ab51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } } }
I20260924 16:06:56.489488 15834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:56.489738 15833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "429605ee3615451fa15b3516db2ab51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "429605ee3615451fa15b3516db2ab51e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44173 } } }
I20260924 16:06:56.489840 15833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:56.489959 15841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:56.493039 15841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:56.498420 15841 catalog_manager.cc:1347] Generated new cluster ID: c5d0cdac8686499abc10084ccceef443
I20260924 16:06:56.498482 15841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:56.516764 15841 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:56.517805 15841 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:56.530033 15841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 429605ee3615451fa15b3516db2ab51e: Generated new TSK 0
I20260924 16:06:56.530931 15841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:06:56.565740 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:56.699102 15853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:56.699453 15853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:56.699514 15853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:56.704823 15853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:56.704895 15853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:56.704973 15853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:06:56.711943 15853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:56.713416 15853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:56.714990 15853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:56.729115 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:56.729156 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:56.729354 15853 server_base.cc:1034] running on GCE node
W20260924 16:06:56.729599 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:56.730149 15853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:56.731105 15853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:56.732324 15853 hybrid_clock.cc:648] HybridClock initialized: now 1790266016732295 us; error 45 us; skew 500 ppm
Sep 24 16:06:56 dist-test-slave-vj6n krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:56.736341 15853 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:06:56.737615 15853 webserver.cc:460] Webserver started at http://127.13.8.65:34223/ using document root <none> and password file <none>
I20260924 16:06:56.738479 15853 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:56.738607 15853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:56.738932 15853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:56.741297 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "b57ee253509544c58cdc8e2be722daf2"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "1207d6f497089a4d6550b33dac5086f7"
server_key_iv: "85efc36af93be8780e91fcbb982cda18"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.741878 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "b57ee253509544c58cdc8e2be722daf2"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "1207d6f497089a4d6550b33dac5086f7"
server_key_iv: "85efc36af93be8780e91fcbb982cda18"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.745613 15853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260924 16:06:56.748201 15868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:56.749441 15853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:06:56.749576 15853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "b57ee253509544c58cdc8e2be722daf2"
format_stamp: "Formatted at 2026-09-24 16:06:56 on dist-test-slave-vj6n"
server_key: "1207d6f497089a4d6550b33dac5086f7"
server_key_iv: "85efc36af93be8780e91fcbb982cda18"
server_key_version: "encryptionkey@0"
I20260924 16:06:56.749688 15853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:56.779196 15853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:56.783006 15853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:56.783349 15853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:56.784313 15853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:56.785670 15853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:56.785770 15853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:56.785920 15853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:56.785990 15853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:56.800369 15853 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:45549
I20260924 16:06:56.800410 15982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:45549 every 8 connection(s)
I20260924 16:06:56.801834 15853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:06:56.802978 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15853
I20260924 16:06:56.803146 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:06:56.803637 13345 external_mini_cluster.cc:1428] Setting key 382dfcdebd22b0674f7a9917867aacdd
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
Sep 24 16:06:56 dist-test-slave-vj6n krb5kdc[15745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:56.825933 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:56.804845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:56065 (local address 127.13.8.126:44173)
0924 16:06:56.805051 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:56.805081 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:56.806713 (+ 1632us) server_negotiation.cc:407] Connection header received
0924 16:06:56.809230 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:56.809234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:56.809288 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:56.809396 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:56.811139 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.811674 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.813611 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.813844 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.813926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:56.819838 (+ 5912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:56.819853 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:56.819863 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:56.819932 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:56.822261 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:56.823215 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:56.823218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:56.823219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:56.823330 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:56.823983 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:56.823986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:56.823987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:56.824256 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:56.824367 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:56.825529 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0924 16:06:56.825675 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 16:06:56.828331 15985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:56.805550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44173 (local address 127.13.8.65:56065)
0924 16:06:56.806440 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:56.806548 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:56.808985 (+ 2437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:56.809665 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:56.809678 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:56.810201 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:56.810957 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:56.810982 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.811817 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.811824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:56.813484 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:56.813497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:56.814012 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:56.814018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:56.814267 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:56.814899 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:56.814923 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:56.819519 (+ 4596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:56.822413 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:56.822430 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:56.822460 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:56.823047 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:56.823462 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:56.823474 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:56.823485 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:56.823836 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:56.824410 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:56.824426 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:56.825092 (+ 666us) client_negotiation.cc:769] Sending connection context
0924 16:06:56.825496 (+ 404us) client_negotiation.cc:240] Negotiation successful
0924 16:06:56.826178 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":207,"threads_started":1}
I20260924 16:06:56.830127 15983 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44173
I20260924 16:06:56.830505 15983 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:56.831264 15983 heartbeater.cc:511] Master 127.13.8.126:44173 requested a full tablet report, sending...
I20260924 16:06:56.833698 15791 ts_manager.cc:194] Registered new tserver with Master: b57ee253509544c58cdc8e2be722daf2 (127.13.8.65:45549)
I20260924 16:06:56.835610 15791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:56065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:06:56.841421 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:56.974927 15988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:56.975353 15988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:56.975492 15988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:56.980696 15988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:56.980813 15988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:56.980973 15988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:06:56.987051 15988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:56.988184 15988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:56.989266 15988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:57.002921 15996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:57.002810 15993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:57.002857 15994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:57.003328 15988 server_base.cc:1034] running on GCE node
I20260924 16:06:57.004027 15988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:57.004935 15988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:57.006134 15988 hybrid_clock.cc:648] HybridClock initialized: now 1790266017006093 us; error 50 us; skew 500 ppm
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:57.010684 15988 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:06:57.012171 15988 webserver.cc:460] Webserver started at http://127.13.8.66:38497/ using document root <none> and password file <none>
I20260924 16:06:57.013118 15988 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:57.013242 15988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:57.013619 15988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:57.016968 15988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "a12e3902bb5644239f604b83c97b1e76"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "d5b5287fc1471686acc4f9bec705ef6e"
server_key_iv: "d6ea6b982916e074a407c54419a257cc"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.017889 15988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "a12e3902bb5644239f604b83c97b1e76"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "d5b5287fc1471686acc4f9bec705ef6e"
server_key_iv: "d6ea6b982916e074a407c54419a257cc"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.024248 15988 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260924 16:06:57.028474 16003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.030910 15988 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:06:57.031100 15988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "a12e3902bb5644239f604b83c97b1e76"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "d5b5287fc1471686acc4f9bec705ef6e"
server_key_iv: "d6ea6b982916e074a407c54419a257cc"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.031263 15988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:57.103670 15988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:57.108451 15988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:57.108814 15988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:57.109799 15988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:57.111219 15988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:57.111316 15988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.111479 15988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:57.111552 15988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.126762 15988 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:34343
I20260924 16:06:57.126816 16119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:34343 every 8 connection(s)
I20260924 16:06:57.128171 15988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:06:57.130738 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 15988
I20260924 16:06:57.130885 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:06:57.131364 13345 external_mini_cluster.cc:1428] Setting key ff9f0255eb6d3cac86eed394ed2fc544
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[15745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:06:57.152504 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:57.130969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:52039 (local address 127.13.8.126:44173)
0924 16:06:57.131148 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:57.131151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:57.132872 (+ 1721us) server_negotiation.cc:407] Connection header received
0924 16:06:57.135058 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:57.135062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:57.135141 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:57.135315 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:57.138245 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.138969 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.140829 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.141174 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.141385 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:57.145968 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:57.145973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:57.145982 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:57.146022 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:57.148106 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:57.149061 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:57.149064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:57.149072 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:57.149200 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:57.149849 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:57.149856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:57.149859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:57.150307 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:57.150506 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:57.151854 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0924 16:06:57.152142 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 16:06:57.155766 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:57.131951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44173 (local address 127.13.8.66:52039)
0924 16:06:57.132664 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:57.132750 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:57.134759 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:57.135588 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:57.135602 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:57.136538 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:57.137951 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:57.137988 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.139199 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.139203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:57.140620 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:57.140630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.141457 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.141463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:57.141744 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:57.142702 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:57.142733 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:57.145615 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:57.148353 (+ 2738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:57.148359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:57.148381 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:57.148880 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:57.149363 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:57.149369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:57.149384 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:57.149688 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:57.150619 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:57.150636 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:57.151261 (+ 625us) client_negotiation.cc:769] Sending connection context
0924 16:06:57.152828 (+ 1567us) client_negotiation.cc:240] Negotiation successful
0924 16:06:57.153524 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":164,"threads_started":1}
I20260924 16:06:57.157967 16120 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44173
I20260924 16:06:57.158517 16120 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:57.159607 16120 heartbeater.cc:511] Master 127.13.8.126:44173 requested a full tablet report, sending...
I20260924 16:06:57.161684 15791 ts_manager.cc:194] Registered new tserver with Master: a12e3902bb5644239f604b83c97b1e76 (127.13.8.66:34343)
I20260924 16:06:57.163142 15791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:52039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:06:57.175395 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:57.341289 16125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:57.341526 16125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:57.341567 16125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:57.345528 16125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:57.345579 16125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:57.345636 16125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:06:57.350715 16125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:57.351675 16125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:57.352629 16125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:57.362246 16131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:57.362299 16130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:57.362251 16133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:57.362396 16125 server_base.cc:1034] running on GCE node
I20260924 16:06:57.362870 16125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:57.363480 16125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:57.364663 16125 hybrid_clock.cc:648] HybridClock initialized: now 1790266017364632 us; error 38 us; skew 500 ppm
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:57.367514 16125 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:06:57.368316 16125 webserver.cc:460] Webserver started at http://127.13.8.67:37927/ using document root <none> and password file <none>
I20260924 16:06:57.368831 16125 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:57.368888 16125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:57.369050 16125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:57.371232 16125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "d0fcd5325eef484891b8ed6bef8844a6"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "84600abfa02afab2e5a965869220375d"
server_key_iv: "387fe5b36607c11bae1326970cbb1051"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.371830 16125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "d0fcd5325eef484891b8ed6bef8844a6"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "84600abfa02afab2e5a965869220375d"
server_key_iv: "387fe5b36607c11bae1326970cbb1051"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.375655 16125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:06:57.378046 16140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.379393 16125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:06:57.379518 16125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "d0fcd5325eef484891b8ed6bef8844a6"
format_stamp: "Formatted at 2026-09-24 16:06:57 on dist-test-slave-vj6n"
server_key: "84600abfa02afab2e5a965869220375d"
server_key_iv: "387fe5b36607c11bae1326970cbb1051"
server_key_version: "encryptionkey@0"
I20260924 16:06:57.379619 16125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:57.412242 16125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:57.415443 16125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:57.415640 16125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:57.416244 16125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:06:57.417091 16125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:06:57.417137 16125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.417207 16125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:06:57.417246 16125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:57.426162 16125 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:46305
I20260924 16:06:57.426189 16255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:46305 every 8 connection(s)
I20260924 16:06:57.427085 16125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:06:57.434922 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16125
I20260924 16:06:57.435056 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:06:57.435390 13345 external_mini_cluster.cc:1428] Setting key ae4a20958a00d098cf834facb80a1d77
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[15745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:57.440968 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:57.428816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:38153 (local address 127.13.8.126:44173)
0924 16:06:57.428972 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:57.428977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:57.429834 (+ 857us) server_negotiation.cc:407] Connection header received
0924 16:06:57.431212 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:57.431215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:57.431260 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:57.431332 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:57.432466 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.432983 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.434063 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.434359 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.434439 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:57.437375 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:57.437378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:57.437381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:57.437405 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:57.438994 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:57.439514 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:57.439518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:57.439519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:57.439598 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:57.439916 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:57.439920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:57.439922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:57.440092 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:57.440191 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:57.440684 (+ 493us) server_negotiation.cc:299] Negotiation successful
0924 16:06:57.440809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 16:06:57.441721 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:57.429214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44173 (local address 127.13.8.67:38153)
0924 16:06:57.429682 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0924 16:06:57.429732 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:06:57.431023 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:06:57.431536 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:06:57.431547 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:06:57.431885 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:06:57.432339 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:57.432351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.433096 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.433098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:57.433958 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:06:57.433966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:06:57.434483 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:06:57.434486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:06:57.434621 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:57.435303 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:06:57.435324 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:06:57.437210 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:06:57.439107 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:57.439113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:57.439127 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:57.439413 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:57.439691 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:06:57.439694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:06:57.439696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:06:57.439827 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:06:57.440190 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:06:57.440198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:06:57.440453 (+ 255us) client_negotiation.cc:769] Sending connection context
0924 16:06:57.440642 (+ 189us) client_negotiation.cc:240] Negotiation successful
0924 16:06:57.440934 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":131,"threads_started":1}
I20260924 16:06:57.442824 16256 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44173
I20260924 16:06:57.443074 16256 heartbeater.cc:464] Registering TS with master...
I20260924 16:06:57.443526 16256 heartbeater.cc:511] Master 127.13.8.126:44173 requested a full tablet report, sending...
I20260924 16:06:57.444490 15791 ts_manager.cc:194] Registered new tserver with Master: d0fcd5325eef484891b8ed6bef8844a6 (127.13.8.67:46305)
I20260924 16:06:57.445004 15791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:38153
I20260924 16:06:57.450292 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 16:06:57.464274 15830 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:06:57.462910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.13.8.126:44173)
0924 16:06:57.463046 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:57.463051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:57.463232 (+ 181us) server_negotiation.cc:407] Connection header received
0924 16:06:57.463339 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:57.463341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:57.463384 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:57.463452 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:57.464150 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33412: BlockingRecv error: recv got EOF from 127.0.0.1:33412 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:06:57.465415 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15853
I20260924 16:06:57.471490 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15988
I20260924 16:06:57.476985 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16125
I20260924 16:06:57.482496 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 15758
2026-09-24T16:06:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2578 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[16268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[16268](info): set up 2 sockets
Sep 24 16:06:57 dist-test-slave-vj6n krb5kdc[16268](info): commencing operation
krb5kdc: starting...
W20260924 16:06:58.527938 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:06:58.650269 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41925
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:06:58.760289 16281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:06:58.760571 16281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:06:58.760609 16281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:06:58.764531 16281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:06:58.764587 16281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:06:58.764606 16281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:06:58.764623 16281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:06:58.764638 16281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:06:58.769764 16281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:06:58.770792 16281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:06:58.771785 16281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:06:58.780179 16286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:58.780244 16289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:06:58.780179 16287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:06:58.780441 16281 server_base.cc:1034] running on GCE node
I20260924 16:06:58.781023 16281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:06:58.782052 16281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:06:58.783211 16281 hybrid_clock.cc:648] HybridClock initialized: now 1790266018783190 us; error 32 us; skew 500 ppm
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266018, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:06:58.792330 16281 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:06:58.793288 16281 webserver.cc:460] Webserver started at http://127.13.8.126:45405/ using document root <none> and password file <none>
I20260924 16:06:58.793862 16281 fs_manager.cc:359] Metadata directory not provided
I20260924 16:06:58.793924 16281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:06:58.794138 16281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:06:58.796204 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "2426061c2035438b8fbb59d67eac061f"
format_stamp: "Formatted at 2026-09-24 16:06:58 on dist-test-slave-vj6n"
server_key: "c7eab1decd44936438019e7d15f116bb"
server_key_iv: "7badc4255d27b85fef2119623d485206"
server_key_version: "encryptionkey@0"
I20260924 16:06:58.796785 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "2426061c2035438b8fbb59d67eac061f"
format_stamp: "Formatted at 2026-09-24 16:06:58 on dist-test-slave-vj6n"
server_key: "c7eab1decd44936438019e7d15f116bb"
server_key_iv: "7badc4255d27b85fef2119623d485206"
server_key_version: "encryptionkey@0"
I20260924 16:06:58.800508 16281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:06:58.802968 16296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:06:58.804251 16281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:06:58.804391 16281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "2426061c2035438b8fbb59d67eac061f"
format_stamp: "Formatted at 2026-09-24 16:06:58 on dist-test-slave-vj6n"
server_key: "c7eab1decd44936438019e7d15f116bb"
server_key_iv: "7badc4255d27b85fef2119623d485206"
server_key_version: "encryptionkey@0"
I20260924 16:06:58.804502 16281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:06:58.824432 16281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:06:58.827733 16281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:06:58.827937 16281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:06:58.835075 16281 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41925
I20260924 16:06:58.835098 16348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41925 every 8 connection(s)
I20260924 16:06:58.836023 16281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:06:58.836463 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16281
I20260924 16:06:58.836606 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:06:58.836898 13345 external_mini_cluster.cc:1428] Setting key edc09bf4e76eb94e122bb4573fdb3c91
I20260924 16:06:58.839018 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:06:58.844506 16349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f: Bootstrap starting.
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:06:58.847101 16349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f: Neither blocks nor log segments found. Creating new log.
I20260924 16:06:58.847934 16349 log.cc:824] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f: Log is configured to *not* fsync() on all Append() calls
I20260924 16:06:58.850121 16349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f: No bootstrap required, opened a new log
I20260924 16:06:58.851485 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:58.838373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.13.8.126:41925)
0924 16:06:58.838794 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:58.838819 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:58.838855 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:06:58.839536 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:58.839555 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:58.839900 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:58.840238 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:58.841062 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:58.842068 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:58.842971 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:58.843436 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:58.843621 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:58.846445 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:58.846451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:58.846464 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:06:58.846498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:58.848417 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:58.849036 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:58.849042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:58.849048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:58.849198 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:06:58.849486 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:06:58.849488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:06:58.849489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:06:58.849899 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:58.850035 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:58.850428 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 16:06:58.850764 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":144,"threads_started":1}
I20260924 16:06:58.853688 16349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } }
I20260924 16:06:58.853919 16349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:06:58.853996 16349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2426061c2035438b8fbb59d67eac061f, State: Initialized, Role: FOLLOWER
I20260924 16:06:58.854432 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:06:58.854517 16349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } }
I20260924 16:06:58.854686 16349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:06:58.854750 16349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:06:58.854844 16349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:06:58.855703 16349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } }
I20260924 16:06:58.856035 16349 leader_election.cc:302] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2426061c2035438b8fbb59d67eac061f; no voters:
I20260924 16:06:58.856390 16349 leader_election.cc:288] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:06:58.856460 16355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:06:58.856662 16355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [term 1 LEADER]: Becoming Leader. State: Replica: 2426061c2035438b8fbb59d67eac061f, State: Running, Role: LEADER
I20260924 16:06:58.856956 16355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } }
I20260924 16:06:58.857717 16349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:06:58.858137 16355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2426061c2035438b8fbb59d67eac061f. Latest consensus state: current_term: 1 leader_uuid: "2426061c2035438b8fbb59d67eac061f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } } }
I20260924 16:06:58.858280 16355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:58.858666 16359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:06:58.858755 16356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2426061c2035438b8fbb59d67eac061f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426061c2035438b8fbb59d67eac061f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41925 } } }
I20260924 16:06:58.858949 16356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f [sys.catalog]: This master's current role is: LEADER
I20260924 16:06:58.861948 16359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:06:58.867815 16359 catalog_manager.cc:1347] Generated new cluster ID: 6014775558364a22baef55fc8a1a4c6e
I20260924 16:06:58.867900 16359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:06:58.875543 16359 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:06:58.876606 16359 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:06:58.880569 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:06:58.874348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.13.8.126:41925)
0924 16:06:58.874478 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:06:58.874483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:06:58.874849 (+ 366us) server_negotiation.cc:407] Connection header received
0924 16:06:58.875042 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:06:58.875045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:06:58.875095 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:06:58.875175 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:06:58.876197 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:58.876932 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:58.878132 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:06:58.878534 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:06:58.878630 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:06:58.879566 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:06:58.879570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:06:58.879573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:06:58.880107 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:06:58.880137 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:06:58.880155 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:06:58.880240 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 16:06:58.880358 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 16:06:58.880431 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":51}
I20260924 16:06:58.889573 16359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2426061c2035438b8fbb59d67eac061f: Generated new TSK 0
I20260924 16:06:58.890539 16359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:06:58.902567 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16281
2026-09-24T16:06:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1418 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16382](info): set up 2 sockets
Sep 24 16:06:58 dist-test-slave-vj6n krb5kdc[16382](info): commencing operation
krb5kdc: starting...
W20260924 16:06:59.944083 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:00.065011 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35183
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:00.179029 16395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:00.179368 16395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:00.179450 16395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:00.183537 16395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:00.183638 16395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:00.183696 16395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:00.183745 16395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:00.183781 16395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:00.189203 16395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:00.190416 16395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:00.191529 16395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:00.200374 16402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.200371 16401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.200659 16395 server_base.cc:1034] running on GCE node
W20260924 16:07:00.200496 16404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.201330 16395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:00.202521 16395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:00.203706 16395 hybrid_clock.cc:648] HybridClock initialized: now 1790266020203693 us; error 26 us; skew 500 ppm
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:00.206425 16395 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:00.207260 16395 webserver.cc:460] Webserver started at http://127.13.8.126:32867/ using document root <none> and password file <none>
I20260924 16:07:00.207784 16395 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:00.207839 16395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:00.208001 16395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:00.209967 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "4fded566d9d14d1caea02ac4aa22bb33"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "14c93354f7264d18366e7203e75f31e5"
server_key_iv: "c7f32504c4fc9490fa5bd9b0ccccdedd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.210536 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "4fded566d9d14d1caea02ac4aa22bb33"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "14c93354f7264d18366e7203e75f31e5"
server_key_iv: "c7f32504c4fc9490fa5bd9b0ccccdedd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.214273 16395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260924 16:07:00.216781 16411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.218128 16395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:00.218314 16395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "4fded566d9d14d1caea02ac4aa22bb33"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "14c93354f7264d18366e7203e75f31e5"
server_key_iv: "c7f32504c4fc9490fa5bd9b0ccccdedd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.218418 16395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:00.247155 16395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:00.250083 16395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:00.250303 16395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:00.257112 16395 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35183
I20260924 16:07:00.257395 16463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35183 every 8 connection(s)
I20260924 16:07:00.258095 16395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:00.261123 16464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:00.262009 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16395
I20260924 16:07:00.262142 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:00.262462 13345 external_mini_cluster.cc:1428] Setting key 3ee3197edd0c67321c445829cd751bcf
I20260924 16:07:00.267261 16464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33: Bootstrap starting.
I20260924 16:07:00.269455 16464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:00.270175 16464 log.cc:824] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:00.272397 16464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33: No bootstrap required, opened a new log
I20260924 16:07:00.275974 16464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } }
I20260924 16:07:00.276218 16464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:00.276353 16464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fded566d9d14d1caea02ac4aa22bb33, State: Initialized, Role: FOLLOWER
I20260924 16:07:00.276372 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.263847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.13.8.126:35183)
0924 16:07:00.264276 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:00.264289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:00.264324 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:07:00.264956 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:00.264972 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:00.265287 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:00.265616 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:00.266468 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.267475 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.268334 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.268810 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.268986 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:00.271715 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:00.271737 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:00.271769 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:00.271802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:00.273474 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.273914 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.273920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.273926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.274047 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.274391 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.274394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.274396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.274795 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:00.275017 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:00.275269 (+ 252us) server_negotiation.cc:299] Negotiation successful
0924 16:07:00.275556 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
I20260924 16:07:00.276891 16464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } }
I20260924 16:07:00.277055 16464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:00.277117 16464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:00.277208 16464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:00.278131 16464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } }
I20260924 16:07:00.278532 16464 leader_election.cc:302] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fded566d9d14d1caea02ac4aa22bb33; no voters:
I20260924 16:07:00.278867 16464 leader_election.cc:288] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:00.278962 16469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:00.279173 16469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [term 1 LEADER]: Becoming Leader. State: Replica: 4fded566d9d14d1caea02ac4aa22bb33, State: Running, Role: LEADER
I20260924 16:07:00.279474 16469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } }
I20260924 16:07:00.280030 16464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:00.281185 16471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fded566d9d14d1caea02ac4aa22bb33. Latest consensus state: current_term: 1 leader_uuid: "4fded566d9d14d1caea02ac4aa22bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } } }
I20260924 16:07:00.281320 16471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:00.281213 16470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fded566d9d14d1caea02ac4aa22bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fded566d9d14d1caea02ac4aa22bb33" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35183 } } }
I20260924 16:07:00.281594 16470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:00.281863 16478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:00.284937 16478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:00.291006 16478 catalog_manager.cc:1347] Generated new cluster ID: 50857f81e4d841cbbc4aee0138e80824
I20260924 16:07:00.291074 16478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:00.308425 16478 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:00.309739 16478 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:00.316216 16478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fded566d9d14d1caea02ac4aa22bb33: Generated new TSK 0
I20260924 16:07:00.317118 16478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:00.345244 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:00.457545 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:00.457863 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:00.457940 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:00.462239 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:00.462339 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:00.462430 16491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:00.467950 16491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:00.469072 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:00.470150 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:00.480260 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.480333 16491 server_base.cc:1034] running on GCE node
W20260924 16:07:00.480268 16497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.480268 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.480892 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:00.481712 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:00.482894 16491 hybrid_clock.cc:648] HybridClock initialized: now 1790266020482877 us; error 29 us; skew 500 ppm
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:00.485620 16491 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:00.486565 16491 webserver.cc:460] Webserver started at http://127.13.8.65:37687/ using document root <none> and password file <none>
I20260924 16:07:00.487092 16491 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:00.487147 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:00.487323 16491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:00.489305 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "f7328d8b2d8644edaef5dbcf8100f018"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "60f70e5b64b0c31870a2013c0b27aeb3"
server_key_iv: "6b5acfa204835b12e4e471c31b2e27c6"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.489800 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "f7328d8b2d8644edaef5dbcf8100f018"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "60f70e5b64b0c31870a2013c0b27aeb3"
server_key_iv: "6b5acfa204835b12e4e471c31b2e27c6"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.493448 16491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:07:00.495908 16506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.497282 16491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:00.497404 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "f7328d8b2d8644edaef5dbcf8100f018"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "60f70e5b64b0c31870a2013c0b27aeb3"
server_key_iv: "6b5acfa204835b12e4e471c31b2e27c6"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.497509 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:00.523679 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:00.526669 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:00.526865 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:00.527479 16491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:00.528307 16491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:00.528352 16491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.528414 16491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:00.528447 16491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.537326 16491 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:33511
I20260924 16:07:00.537349 16620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:33511 every 8 connection(s)
I20260924 16:07:00.538199 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:00.542042 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16491
I20260924 16:07:00.542193 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:00.542517 13345 external_mini_cluster.cc:1428] Setting key 4add24714e9ae9325a882b16210d8499
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:00.553043 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.540163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:56105 (local address 127.13.8.126:35183)
0924 16:07:00.540300 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:00.540303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:00.541183 (+ 880us) server_negotiation.cc:407] Connection header received
0924 16:07:00.542729 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:00.542732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:00.542793 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:00.542900 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:00.544077 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.544602 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.545742 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.545978 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.546089 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:00.549105 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:00.549108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:00.549110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:00.549134 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:00.550675 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.551368 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.551371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.551372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.551444 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.551866 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.551868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.551869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.552038 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:00.552110 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:00.552824 (+ 714us) server_negotiation.cc:299] Negotiation successful
0924 16:07:00.552922 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:00.554382 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.540540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35183 (local address 127.13.8.65:56105)
0924 16:07:00.541032 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:00.541076 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:00.542481 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:00.543086 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:00.543093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:00.543449 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:00.543933 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:00.543947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.544735 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.544745 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:00.545618 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:00.545627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.546131 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.546136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:00.546371 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:00.546960 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:00.546977 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:00.548920 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:00.550800 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:00.550808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:00.550832 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:00.551261 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:00.551545 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:00.551549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:00.551551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:00.551767 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:00.552156 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:00.552165 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:00.552544 (+ 379us) client_negotiation.cc:769] Sending connection context
0924 16:07:00.552806 (+ 262us) client_negotiation.cc:240] Negotiation successful
0924 16:07:00.553225 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260924 16:07:00.555768 16622 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35183
I20260924 16:07:00.556008 16622 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:00.556715 16622 heartbeater.cc:511] Master 127.13.8.126:35183 requested a full tablet report, sending...
I20260924 16:07:00.558414 16428 ts_manager.cc:194] Registered new tserver with Master: f7328d8b2d8644edaef5dbcf8100f018 (127.13.8.65:33511)
I20260924 16:07:00.559646 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:56105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:00.573359 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:00.690544 16627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:00.690857 16627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:00.690933 16627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:00.695016 16627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:00.695113 16627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:00.695223 16627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:00.700850 16627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:00.702036 16627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:00.703132 16627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:00.713150 16632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.713189 16635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.713157 16633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.713364 16627 server_base.cc:1034] running on GCE node
I20260924 16:07:00.713825 16627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:00.714529 16627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:00.715744 16627 hybrid_clock.cc:648] HybridClock initialized: now 1790266020715712 us; error 52 us; skew 500 ppm
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:00.718667 16627 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:00.719653 16627 webserver.cc:460] Webserver started at http://127.13.8.66:42475/ using document root <none> and password file <none>
I20260924 16:07:00.720239 16627 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:00.720330 16627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:00.720566 16627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:00.722751 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "a7c69f85225a417b8519850fe99add17"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "c65dcd12de82dfef4fcf508c28a1cef9"
server_key_iv: "ae1635e9005a796d80e968f0a5b173c4"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.723429 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "a7c69f85225a417b8519850fe99add17"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "c65dcd12de82dfef4fcf508c28a1cef9"
server_key_iv: "ae1635e9005a796d80e968f0a5b173c4"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.727350 16627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260924 16:07:00.729759 16642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.731012 16627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:00.731104 16627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "a7c69f85225a417b8519850fe99add17"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "c65dcd12de82dfef4fcf508c28a1cef9"
server_key_iv: "ae1635e9005a796d80e968f0a5b173c4"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.731184 16627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:00.754554 16627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:00.759344 16627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:00.759586 16627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:00.760375 16627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:00.761360 16627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:00.761427 16627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.761478 16627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:00.761505 16627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.770406 16627 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:35213
I20260924 16:07:00.770495 16756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:35213 every 8 connection(s)
I20260924 16:07:00.771355 16627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:00.780332 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16627
I20260924 16:07:00.780453 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:00.780797 13345 external_mini_cluster.cc:1428] Setting key ec77e738f4a8f5c565e57aa6028be4d3
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:00.785951 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.773283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:36131 (local address 127.13.8.126:35183)
0924 16:07:00.773446 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:00.773450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:00.774291 (+ 841us) server_negotiation.cc:407] Connection header received
0924 16:07:00.775716 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:00.775719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:00.775776 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:00.775883 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:00.777093 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.777646 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.778896 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.779120 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.779234 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:00.782150 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:00.782154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:00.782156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:00.782183 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:00.783564 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.784224 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.784229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.784232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.784331 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:00.784741 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:00.784746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:00.784751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:00.784997 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:00.785108 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:00.785658 (+ 550us) server_negotiation.cc:299] Negotiation successful
0924 16:07:00.785787 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 16:07:00.786979 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.773623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35183 (local address 127.13.8.66:36131)
0924 16:07:00.774107 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:00.774151 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:00.775481 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:00.776082 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:00.776090 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:00.776465 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:00.776941 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:00.776955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.777781 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.777790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:00.778765 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:00.778775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.779279 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.779283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:00.779461 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:00.780103 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:00.780127 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:00.781978 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:00.783681 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:00.783690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:00.783709 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:00.784115 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:00.784467 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:00.784469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:00.784471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:00.784629 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:00.785117 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:00.785122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:00.785366 (+ 244us) client_negotiation.cc:769] Sending connection context
0924 16:07:00.785639 (+ 273us) client_negotiation.cc:240] Negotiation successful
0924 16:07:00.785995 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260924 16:07:00.788189 16757 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35183
I20260924 16:07:00.788470 16757 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:00.789089 16757 heartbeater.cc:511] Master 127.13.8.126:35183 requested a full tablet report, sending...
I20260924 16:07:00.790302 16428 ts_manager.cc:194] Registered new tserver with Master: a7c69f85225a417b8519850fe99add17 (127.13.8.66:35213)
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:00.791110 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:36131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:00.809927 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:00.921351 16763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:00.921694 16763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:00.921739 16763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:00.925855 16763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:00.925913 16763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:00.926011 16763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:00.931154 16763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:00.932174 16763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:00.933187 16763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:00.944072 16768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.944044 16771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:00.944060 16769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:00.944478 16763 server_base.cc:1034] running on GCE node
I20260924 16:07:00.944986 16763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:00.945648 16763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:00.946838 16763 hybrid_clock.cc:648] HybridClock initialized: now 1790266020946810 us; error 37 us; skew 500 ppm
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:00.949558 16763 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:00.950528 16763 webserver.cc:460] Webserver started at http://127.13.8.67:40567/ using document root <none> and password file <none>
I20260924 16:07:00.951098 16763 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:00.951190 16763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:00.951443 16763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:00.953493 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "a52787f55ce749a88414190620ef0b1a"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "4610d2719b773e216bfaebe34b51937c"
server_key_iv: "fd93522eee90307c243f71019991d9dd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.954046 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "a52787f55ce749a88414190620ef0b1a"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "4610d2719b773e216bfaebe34b51937c"
server_key_iv: "fd93522eee90307c243f71019991d9dd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.957715 16763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:00.960151 16778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.961433 16763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:00.961603 16763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "a52787f55ce749a88414190620ef0b1a"
format_stamp: "Formatted at 2026-09-24 16:07:00 on dist-test-slave-vj6n"
server_key: "4610d2719b773e216bfaebe34b51937c"
server_key_iv: "fd93522eee90307c243f71019991d9dd"
server_key_version: "encryptionkey@0"
I20260924 16:07:00.961789 16763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:00.976629 16763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:00.979823 16763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:00.979995 16763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:00.980600 16763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:00.981480 16763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:00.981561 16763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.981632 16763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:00.981680 16763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:00.991175 16763 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:38593
I20260924 16:07:00.991201 16891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:38593 every 8 connection(s)
I20260924 16:07:00.992151 16763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:00.996129 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16763
I20260924 16:07:00.996305 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:00.996636 13345 external_mini_cluster.cc:1428] Setting key 6c3af85bb15d140b41d0c1c9617bb956
Sep 24 16:07:00 dist-test-slave-vj6n krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266020, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:01.006865 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.994295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:59603 (local address 127.13.8.126:35183)
0924 16:07:00.994465 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:00.994469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:00.995379 (+ 910us) server_negotiation.cc:407] Connection header received
0924 16:07:00.997213 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:00.997216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:00.997273 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:00.997377 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:00.998884 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.999398 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:01.000547 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:01.000826 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:01.000938 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:01.003461 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:01.003464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:01.003466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:01.003489 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:01.004956 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:01.005415 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:01.005418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:01.005419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:01.005492 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:01.005799 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:01.005801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:01.005802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:01.005974 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:01.006045 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:01.006635 (+ 590us) server_negotiation.cc:299] Negotiation successful
0924 16:07:01.006743 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 16:07:01.007683 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:00.994699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35183 (local address 127.13.8.67:59603)
0924 16:07:00.995204 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:00.995249 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:00.996995 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:00.997607 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:00.997617 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:00.998051 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:00.998734 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:00.998751 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:00.999528 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:00.999532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:01.000422 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:01.000431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:01.001001 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:01.001006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:01.001174 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:01.001788 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:01.001822 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:01.003308 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:01.005055 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:01.005060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:01.005073 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:01.005325 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:01.005566 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:01.005569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:01.005570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:01.005710 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:01.006058 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:01.006063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:01.006384 (+ 321us) client_negotiation.cc:769] Sending connection context
0924 16:07:01.006543 (+ 159us) client_negotiation.cc:240] Negotiation successful
0924 16:07:01.006822 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":123,"threads_started":1}
I20260924 16:07:01.008899 16892 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35183
I20260924 16:07:01.009155 16892 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:01.009763 16892 heartbeater.cc:511] Master 127.13.8.126:35183 requested a full tablet report, sending...
I20260924 16:07:01.010987 16428 ts_manager.cc:194] Registered new tserver with Master: a52787f55ce749a88414190620ef0b1a (127.13.8.67:38593)
I20260924 16:07:01.011533 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:59603
I20260924 16:07:01.022641 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 16:07:01 dist-test-slave-vj6n krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266021, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 16:07:01 dist-test-slave-vj6n krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266021, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:01.048631 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:01.039741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.13.8.126:35183)
0924 16:07:01.039936 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:01.039941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:01.040104 (+ 163us) server_negotiation.cc:407] Connection header received
0924 16:07:01.040286 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:01.040288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:01.040333 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:01.040419 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:01.041133 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:01.041668 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:01.042593 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:01.042818 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:01.042932 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:01.045382 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:01.045385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:01.045387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:01.045411 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:01.046874 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:01.047297 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:01.047301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:01.047303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:01.047389 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:01.047704 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:01.047707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:01.047709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:01.047940 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:01.048090 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:01.048320 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 16:07:01.048456 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260924 16:07:01.049054 16428 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57092
I20260924 16:07:01.049948 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16491
I20260924 16:07:01.056418 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16627
I20260924 16:07:01.061955 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16763
I20260924 16:07:01.067759 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16395
2026-09-24T16:07:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2167 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:01 dist-test-slave-vj6n krb5kdc[16905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:01 dist-test-slave-vj6n krb5kdc[16905](info): set up 2 sockets
Sep 24 16:07:01 dist-test-slave-vj6n krb5kdc[16905](info): commencing operation
krb5kdc: starting...
W20260924 16:07:02.107215 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:02.235730 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:45783
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:45783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:02.348788 16918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:02.349069 16918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:02.349148 16918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:02.353076 16918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:02.353173 16918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:02.353206 16918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:02.353267 16918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:02.353303 16918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:02.358527 16918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:45783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:45783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:02.359572 16918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:02.360608 16918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:02.369174 16926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:02.369175 16923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:02.369228 16924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:02.369396 16918 server_base.cc:1034] running on GCE node
I20260924 16:07:02.370136 16918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:02.371277 16918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:02.372439 16918 hybrid_clock.cc:648] HybridClock initialized: now 1790266022372426 us; error 25 us; skew 500 ppm
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:02.375067 16918 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:02.375895 16918 webserver.cc:460] Webserver started at http://127.13.8.126:33767/ using document root <none> and password file <none>
I20260924 16:07:02.376464 16918 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:02.376524 16918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:02.376775 16918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:02.378798 16918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "67bd2c741a0641998b95a74da56ee155"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "b20dd2bfbe4a768123db5d324b1747d3"
server_key_iv: "637dddbe873b5ff28df98c28d476da9f"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.379340 16918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "67bd2c741a0641998b95a74da56ee155"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "b20dd2bfbe4a768123db5d324b1747d3"
server_key_iv: "637dddbe873b5ff28df98c28d476da9f"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.382984 16918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:07:02.385234 16933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.386490 16918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:02.386592 16918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "67bd2c741a0641998b95a74da56ee155"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "b20dd2bfbe4a768123db5d324b1747d3"
server_key_iv: "637dddbe873b5ff28df98c28d476da9f"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.386715 16918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:02.404399 16918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:02.407716 16918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:02.407934 16918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:02.415184 16918 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:45783
I20260924 16:07:02.415205 16985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:45783 every 8 connection(s)
I20260924 16:07:02.416141 16918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:02.419133 16986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:02.421734 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 16918
I20260924 16:07:02.421821 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:02.422092 13345 external_mini_cluster.cc:1428] Setting key 9827f89594605cab09f17718613d6df9
I20260924 16:07:02.425228 16986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155: Bootstrap starting.
I20260924 16:07:02.427987 16986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:02.428846 16986 log.cc:824] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:02.430948 16986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155: No bootstrap required, opened a new log
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:02.434157 16986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } }
I20260924 16:07:02.434542 16986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:02.434620 16986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bd2c741a0641998b95a74da56ee155, State: Initialized, Role: FOLLOWER
I20260924 16:07:02.435134 16986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } }
I20260924 16:07:02.435338 16986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:02.435412 16986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:02.435524 16986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:02.436704 16986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } }
I20260924 16:07:02.437068 16986 leader_election.cc:302] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67bd2c741a0641998b95a74da56ee155; no voters:
I20260924 16:07:02.437408 16986 leader_election.cc:288] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:02.437517 16991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:02.437604 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:02.423437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.13.8.126:45783)
0924 16:07:02.423805 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:02.423825 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:02.423870 (+ 45us) server_negotiation.cc:407] Connection header received
0924 16:07:02.424523 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:02.424540 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:02.424907 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:02.425253 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:02.426192 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.427255 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.428454 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.428910 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.429095 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:02.431880 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:02.431885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:02.431897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:02.431923 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:02.434190 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.434802 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.434808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.434813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.434961 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.435272 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.435276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.435278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.435834 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:02.436067 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:02.436247 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 16:07:02.436605 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":118,"threads_started":1}
I20260924 16:07:02.437773 16991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [term 1 LEADER]: Becoming Leader. State: Replica: 67bd2c741a0641998b95a74da56ee155, State: Running, Role: LEADER
I20260924 16:07:02.438102 16991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } }
I20260924 16:07:02.438783 16986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:02.439781 16992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67bd2c741a0641998b95a74da56ee155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } } }
I20260924 16:07:02.440188 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67bd2c741a0641998b95a74da56ee155. Latest consensus state: current_term: 1 leader_uuid: "67bd2c741a0641998b95a74da56ee155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67bd2c741a0641998b95a74da56ee155" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45783 } } }
I20260924 16:07:02.440420 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:02.440641 16992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:02.441187 17001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:02.443861 17001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:02.449147 17001 catalog_manager.cc:1347] Generated new cluster ID: f7df8fae94ae43e6ba38267ad5afc85e
I20260924 16:07:02.449208 17001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:02.470240 17001 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:02.471657 17001 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:02.488821 17001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67bd2c741a0641998b95a74da56ee155: Generated new TSK 0
I20260924 16:07:02.489730 17001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:02.526005 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:02.658011 17012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:02.658329 17012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:02.658373 17012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:02.662258 17012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:02.662303 17012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:02.662358 17012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:02.667477 17012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:02.668442 17012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:02.669394 17012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:02.678918 17020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:02.678960 17018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:02.678925 17017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:02.679164 17012 server_base.cc:1034] running on GCE node
I20260924 16:07:02.679656 17012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:02.680299 17012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:02.681483 17012 hybrid_clock.cc:648] HybridClock initialized: now 1790266022681464 us; error 30 us; skew 500 ppm
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:02.684150 17012 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:02.684962 17012 webserver.cc:460] Webserver started at http://127.13.8.65:36029/ using document root <none> and password file <none>
I20260924 16:07:02.685513 17012 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:02.685603 17012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:02.685830 17012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:02.687850 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "25337880d7d840c1acdc45bab623e901"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "4862e5d1781de835a900546b256457e8"
server_key_iv: "b16e512e9107b690a08a76e6b002a9ad"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.688396 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "25337880d7d840c1acdc45bab623e901"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "4862e5d1781de835a900546b256457e8"
server_key_iv: "b16e512e9107b690a08a76e6b002a9ad"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.692245 17012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260924 16:07:02.694564 17027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.695731 17012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:02.695819 17012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "25337880d7d840c1acdc45bab623e901"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "4862e5d1781de835a900546b256457e8"
server_key_iv: "b16e512e9107b690a08a76e6b002a9ad"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.695896 17012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:02.714349 17012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:02.717293 17012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:02.717439 17012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:02.717967 17012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:02.718751 17012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:02.718796 17012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.718834 17012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:02.718849 17012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.728170 17012 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:40397
I20260924 16:07:02.728192 17141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:40397 every 8 connection(s)
I20260924 16:07:02.729113 17012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:02.733063 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17012
I20260924 16:07:02.733160 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:02.733487 13345 external_mini_cluster.cc:1428] Setting key 6248cffb5237c21f832a7e410f4e7dc2
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:02.743970 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:02.731021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:34985 (local address 127.13.8.126:45783)
0924 16:07:02.731187 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:02.731192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:02.732010 (+ 818us) server_negotiation.cc:407] Connection header received
0924 16:07:02.733477 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:02.733480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:02.733541 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:02.733633 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:02.735004 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.735799 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.736941 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.737176 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.737288 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:02.740156 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:02.740162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:02.740168 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:02.740203 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:02.741835 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.742339 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.742343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.742344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.742424 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.742767 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.742774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.742777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.743020 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:02.743147 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:02.743600 (+ 453us) server_negotiation.cc:299] Negotiation successful
0924 16:07:02.743749 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 16:07:02.744709 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:02.731430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45783 (local address 127.13.8.65:34985)
0924 16:07:02.731865 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:02.731906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:02.733247 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:02.733804 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:02.733811 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:02.734312 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:02.734886 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:02.734902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.735924 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.735927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:02.736813 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:02.736822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.737324 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.737328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:02.737553 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:02.738186 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:02.738212 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:02.739985 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:02.741944 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:02.741949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:02.741960 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:02.742189 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:02.742524 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:02.742527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:02.742528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:02.742658 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:02.743130 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:02.743134 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:02.743349 (+ 215us) client_negotiation.cc:769] Sending connection context
0924 16:07:02.743487 (+ 138us) client_negotiation.cc:240] Negotiation successful
0924 16:07:02.743803 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
I20260924 16:07:02.746160 17142 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45783
I20260924 16:07:02.746465 17142 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:02.747064 17142 heartbeater.cc:511] Master 127.13.8.126:45783 requested a full tablet report, sending...
I20260924 16:07:02.748638 16950 ts_manager.cc:194] Registered new tserver with Master: 25337880d7d840c1acdc45bab623e901 (127.13.8.65:40397)
I20260924 16:07:02.750370 16950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:34985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:02.761264 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:02.875111 17147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:02.875375 17147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:02.875414 17147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:02.879478 17147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:02.879531 17147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:02.879591 17147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:02.884680 17147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:02.885730 17147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:02.886808 17147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:02.896613 17153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:02.896721 17147 server_base.cc:1034] running on GCE node
W20260924 16:07:02.896631 17154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:02.896907 17156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:02.897329 17147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:02.898000 17147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:02.899189 17147 hybrid_clock.cc:648] HybridClock initialized: now 1790266022899159 us; error 38 us; skew 500 ppm
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:02.901927 17147 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:02.902928 17147 webserver.cc:460] Webserver started at http://127.13.8.66:46609/ using document root <none> and password file <none>
I20260924 16:07:02.903493 17147 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:02.903632 17147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:02.903868 17147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:02.905879 17147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "8ab2c10a25ba4562965d57e14667923d"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "402c67a5497fb580a7ad77baac456f2e"
server_key_iv: "c37778af3281af773e8b17fc5174e653"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.906482 17147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "8ab2c10a25ba4562965d57e14667923d"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "402c67a5497fb580a7ad77baac456f2e"
server_key_iv: "c37778af3281af773e8b17fc5174e653"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.910410 17147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:07:02.912985 17163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.914273 17147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:02.914409 17147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "8ab2c10a25ba4562965d57e14667923d"
format_stamp: "Formatted at 2026-09-24 16:07:02 on dist-test-slave-vj6n"
server_key: "402c67a5497fb580a7ad77baac456f2e"
server_key_iv: "c37778af3281af773e8b17fc5174e653"
server_key_version: "encryptionkey@0"
I20260924 16:07:02.914518 17147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:02.933539 17147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:02.936519 17147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:02.936724 17147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:02.937315 17147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:02.938139 17147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:02.938205 17147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.938316 17147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:02.938366 17147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:02.947767 17147 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:41427
I20260924 16:07:02.947818 17276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:41427 every 8 connection(s)
I20260924 16:07:02.949039 17147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:02.957960 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17147
I20260924 16:07:02.958062 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:02.958436 13345 external_mini_cluster.cc:1428] Setting key 6a064d8f63559faa8d875d90866f4504
Sep 24 16:07:02 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:02.963370 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:02.951127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:34639 (local address 127.13.8.126:45783)
0924 16:07:02.951274 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:02.951279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:02.952186 (+ 907us) server_negotiation.cc:407] Connection header received
0924 16:07:02.953538 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:02.953541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:02.953611 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:02.953684 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:02.954912 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.955449 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.956567 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.956799 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.956889 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:02.959540 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:02.959544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:02.959546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:02.959572 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:02.961059 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.961693 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.961698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.961700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.961799 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:02.962116 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:02.962120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:02.962125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:02.962403 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:02.962509 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:02.962972 (+ 463us) server_negotiation.cc:299] Negotiation successful
0924 16:07:02.963120 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:07:02.964008 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:02.951593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45783 (local address 127.13.8.66:34639)
0924 16:07:02.952042 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:02.952085 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:02.953331 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:02.953849 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:02.953856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:02.954257 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:02.954766 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:02.954779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.955560 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.955562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:02.956443 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:02.956452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:02.956934 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:02.956936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:02.957076 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:02.957734 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:02.957750 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:02.959379 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:02.961177 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:02.961186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:02.961205 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:02.961577 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:02.961897 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:02.961899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:02.961900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:02.962026 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:02.962500 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:02.962505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:02.962734 (+ 229us) client_negotiation.cc:769] Sending connection context
0924 16:07:02.962885 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 16:07:02.963120 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260924 16:07:02.965243 17277 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45783
I20260924 16:07:02.965480 17277 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:02.965952 17277 heartbeater.cc:511] Master 127.13.8.126:45783 requested a full tablet report, sending...
I20260924 16:07:02.967118 16950 ts_manager.cc:194] Registered new tserver with Master: 8ab2c10a25ba4562965d57e14667923d (127.13.8.66:41427)
I20260924 16:07:02.967676 16950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:34639
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:02.985564 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:03.098136 17282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:03.098428 17282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:03.098513 17282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:03.102756 17282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:03.102820 17282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:03.102931 17282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:03.108333 17282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:03.109544 17282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:03.110675 17282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:03.120553 17291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:03.120558 17288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:03.120592 17282 server_base.cc:1034] running on GCE node
W20260924 16:07:03.120553 17289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:03.121306 17282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:03.121891 17282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:03.123041 17282 hybrid_clock.cc:648] HybridClock initialized: now 1790266023122996 us; error 56 us; skew 500 ppm
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266023, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:03.125824 17282 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:03.126751 17282 webserver.cc:460] Webserver started at http://127.13.8.67:43275/ using document root <none> and password file <none>
I20260924 16:07:03.127329 17282 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:03.127419 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:03.127660 17282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:03.129714 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "ca03214446a847298c57274ffa9afe35"
format_stamp: "Formatted at 2026-09-24 16:07:03 on dist-test-slave-vj6n"
server_key: "b205dc349ed35209c23990b909e26a87"
server_key_iv: "890d8ade2be258d533f5376a5b47ac27"
server_key_version: "encryptionkey@0"
I20260924 16:07:03.130343 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "ca03214446a847298c57274ffa9afe35"
format_stamp: "Formatted at 2026-09-24 16:07:03 on dist-test-slave-vj6n"
server_key: "b205dc349ed35209c23990b909e26a87"
server_key_iv: "890d8ade2be258d533f5376a5b47ac27"
server_key_version: "encryptionkey@0"
I20260924 16:07:03.134142 17282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:07:03.136552 17298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:03.137728 17282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:03.137861 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "ca03214446a847298c57274ffa9afe35"
format_stamp: "Formatted at 2026-09-24 16:07:03 on dist-test-slave-vj6n"
server_key: "b205dc349ed35209c23990b909e26a87"
server_key_iv: "890d8ade2be258d533f5376a5b47ac27"
server_key_version: "encryptionkey@0"
I20260924 16:07:03.137969 17282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:03.150928 17282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:03.153666 17282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:03.153867 17282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:03.154487 17282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:03.155297 17282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:03.155369 17282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:03.155440 17282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:03.155483 17282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:03.164317 17282 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:46235
I20260924 16:07:03.164338 17411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:46235 every 8 connection(s)
I20260924 16:07:03.165225 17282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:03.171689 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17282
I20260924 16:07:03.171831 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:03.172119 13345 external_mini_cluster.cc:1428] Setting key 982ff61eb4f97823e813ba9323c840ad
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266023, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:03.179388 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:03.167443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:58587 (local address 127.13.8.126:45783)
0924 16:07:03.167599 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:03.167604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:03.168562 (+ 958us) server_negotiation.cc:407] Connection header received
0924 16:07:03.169862 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:03.169865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:03.169912 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:03.169995 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:03.171249 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.171926 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.173349 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.173577 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.173687 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:03.176083 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:03.176087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:03.176090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:03.176123 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:03.177424 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.177936 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.177938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.177940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.178015 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.178375 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.178377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.178378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.178548 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:03.178668 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:03.179180 (+ 512us) server_negotiation.cc:299] Negotiation successful
0924 16:07:03.179276 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:07:03.180187 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:03.167944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45783 (local address 127.13.8.67:58587)
0924 16:07:03.168405 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:03.168448 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:03.169655 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:03.170154 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:03.170161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:03.170603 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:03.171066 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:03.171079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.172101 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.172104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:03.173213 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:03.173223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.173726 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.173730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:03.173904 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:03.174515 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:03.174532 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:03.175922 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:03.177547 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:03.177551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:03.177563 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:03.177833 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:03.178102 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:03.178105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:03.178106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:03.178296 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:03.178694 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:03.178699 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:03.178936 (+ 237us) client_negotiation.cc:769] Sending connection context
0924 16:07:03.179086 (+ 150us) client_negotiation.cc:240] Negotiation successful
0924 16:07:03.179350 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260924 16:07:03.181318 17412 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45783
I20260924 16:07:03.181562 17412 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:03.182029 17412 heartbeater.cc:511] Master 127.13.8.126:45783 requested a full tablet report, sending...
I20260924 16:07:03.183085 16950 ts_manager.cc:194] Registered new tserver with Master: ca03214446a847298c57274ffa9afe35 (127.13.8.67:46235)
I20260924 16:07:03.183701 16950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:58587
I20260924 16:07:03.185860 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:07:03.211575 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:03.202121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.13.8.65:40397)
0924 16:07:03.202437 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:03.202453 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:03.202479 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:07:03.202592 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:03.202600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:03.202811 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:03.202982 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:03.203646 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.204420 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.205282 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.205657 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.205777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:03.208308 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:03.208317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:03.208331 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:03.208353 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:03.209700 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.210154 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.210167 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.210168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.210298 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.210573 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.210577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.210579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.210829 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:03.211072 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:03.211246 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 16:07:03.211398 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":115,"threads_started":1}
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:03.222143 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:03.214436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.13.8.126:45783)
0924 16:07:03.214582 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:03.214586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:03.214603 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:07:03.214730 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:03.214732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:03.214773 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:03.214839 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:03.215378 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.215870 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.216608 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:03.216816 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:03.216887 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:03.219318 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:03.219320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:03.219322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:03.219345 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:03.220715 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.221082 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.221084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.221085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.221155 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:03.221380 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:03.221382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:03.221383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:03.221568 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:03.221662 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:03.221856 (+ 194us) server_negotiation.cc:299] Negotiation successful
0924 16:07:03.221937 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260924 16:07:03.222570 16950 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216
I20260924 16:07:03.223148 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17012
I20260924 16:07:03.229730 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17147
I20260924 16:07:03.235857 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17282
I20260924 16:07:03.241523 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 16918
2026-09-24T16:07:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2172 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[17425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[17425](info): set up 2 sockets
Sep 24 16:07:03 dist-test-slave-vj6n krb5kdc[17425](info): commencing operation
krb5kdc: starting...
W20260924 16:07:04.363277 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:04 dist-test-slave-vj6n krb5kdc[17425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:04.483040 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40007
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:04.594784 17438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:04.595057 17438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:04.595103 17438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:04.599087 17438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:04.599143 17438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:04.599162 17438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:04.599179 17438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:04.599193 17438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:04.604418 17438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:04.605427 17438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:04.606458 17438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:04.614702 17443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:04.614722 17438 server_base.cc:1034] running on GCE node
W20260924 16:07:04.614709 17444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:04.614738 17446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:04.615504 17438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:04.616667 17438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:04.617830 17438 hybrid_clock.cc:648] HybridClock initialized: now 1790266024617808 us; error 37 us; skew 500 ppm
Sep 24 16:07:04 dist-test-slave-vj6n krb5kdc[17425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:04.634063 17438 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:04.635852 17438 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:04.635964 17438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:04.636225 17438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:04.638746 17438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "d5c9822e3ac347feb48c91253237d195"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "a58dd34d6bc8a1c5fa6e4c7acec5a325"
server_key_iv: "fb489634f287b2c77c1ef91b4a0b525f"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.639379 17438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "d5c9822e3ac347feb48c91253237d195"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "a58dd34d6bc8a1c5fa6e4c7acec5a325"
server_key_iv: "fb489634f287b2c77c1ef91b4a0b525f"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.643458 17438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260924 16:07:04.646193 17452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:04.647670 17438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:04.647836 17438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "d5c9822e3ac347feb48c91253237d195"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "a58dd34d6bc8a1c5fa6e4c7acec5a325"
server_key_iv: "fb489634f287b2c77c1ef91b4a0b525f"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.647959 17438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:04.662451 17438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:04.665307 17438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:04.665509 17438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:04.672582 17438 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:40007
I20260924 16:07:04.672901 17504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:40007 every 8 connection(s)
I20260924 16:07:04.673332 17438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:04.676239 17505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:04.679297 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17438
I20260924 16:07:04.679438 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:04.679726 13345 external_mini_cluster.cc:1428] Setting key 8fa7f96741e28befd0446650e4ef890f
I20260924 16:07:04.682677 17505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: Bootstrap starting.
I20260924 16:07:04.684916 17505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:04.685652 17505 log.cc:824] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:04.687664 17505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: No bootstrap required, opened a new log
Sep 24 16:07:04 dist-test-slave-vj6n krb5kdc[17425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:04.691440 17505 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } }
I20260924 16:07:04.691787 17505 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:04.691877 17505 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c9822e3ac347feb48c91253237d195, State: Initialized, Role: FOLLOWER
I20260924 16:07:04.692507 17505 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } }
I20260924 16:07:04.692668 17505 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:04.692752 17505 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:04.692855 17505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:04.693743 17505 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } }
I20260924 16:07:04.694150 17505 leader_election.cc:302] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5c9822e3ac347feb48c91253237d195; no voters:
I20260924 16:07:04.694552 17505 leader_election.cc:288] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:04.694641 17510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:04.694849 17510 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [term 1 LEADER]: Becoming Leader. State: Replica: d5c9822e3ac347feb48c91253237d195, State: Running, Role: LEADER
I20260924 16:07:04.695153 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:04.681175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.13.8.126:40007)
0924 16:07:04.681633 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:04.681647 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:04.681683 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:07:04.682343 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:04.682362 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:04.682695 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:04.683076 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:04.683887 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.684864 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.685752 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.686288 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.686450 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:04.689162 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:04.689167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:04.689180 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:04.689207 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:04.691376 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:04.691955 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:04.691963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:04.691972 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:04.692233 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:04.692712 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:04.692717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:04.692719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:04.693236 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:04.693445 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:04.693752 (+ 307us) server_negotiation.cc:299] Negotiation successful
0924 16:07:04.694154 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":130,"threads_started":1}
I20260924 16:07:04.695219 17510 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } }
I20260924 16:07:04.695858 17505 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:04.696815 17511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5c9822e3ac347feb48c91253237d195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } } }
I20260924 16:07:04.697006 17511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:04.697441 17519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:04.697767 17512 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5c9822e3ac347feb48c91253237d195. Latest consensus state: current_term: 1 leader_uuid: "d5c9822e3ac347feb48c91253237d195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c9822e3ac347feb48c91253237d195" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40007 } } }
I20260924 16:07:04.697876 17512 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:04.700557 17519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:04.706090 17519 catalog_manager.cc:1347] Generated new cluster ID: 9f2aaeb27be14768ac2dba4401dbaeba
I20260924 16:07:04.706163 17519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:04.711709 17519 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:04.712939 17519 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:04.719547 17519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: Generated new TSK 0
I20260924 16:07:04.720278 17519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:04.751273 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:04.867236 17531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:04.867588 17531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:04.867682 17531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:04.871969 17531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:04.872072 17531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:04.872193 17531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:04.877754 17531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:04.878965 17531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:04.880048 17531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:04.889575 17536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:04.889623 17531 server_base.cc:1034] running on GCE node
W20260924 16:07:04.889575 17539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:04.889576 17537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:04.890326 17531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:04.890975 17531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:04.892162 17531 hybrid_clock.cc:648] HybridClock initialized: now 1790266024892136 us; error 46 us; skew 500 ppm
Sep 24 16:07:04 dist-test-slave-vj6n krb5kdc[17425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:04.895126 17531 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:04.895948 17531 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:04.896036 17531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:04.896279 17531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:04.898653 17531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "a2895615829442b997615d1d0f3cc11e"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "88aa8dd1d2bb627f2212ddf37ca076c6"
server_key_iv: "496e9b58e7873866b8e05eef012a7939"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.899200 17531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "a2895615829442b997615d1d0f3cc11e"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "88aa8dd1d2bb627f2212ddf37ca076c6"
server_key_iv: "496e9b58e7873866b8e05eef012a7939"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.902936 17531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:07:04.905370 17545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:04.906795 17531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:07:04.906946 17531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "a2895615829442b997615d1d0f3cc11e"
format_stamp: "Formatted at 2026-09-24 16:07:04 on dist-test-slave-vj6n"
server_key: "88aa8dd1d2bb627f2212ddf37ca076c6"
server_key_iv: "496e9b58e7873866b8e05eef012a7939"
server_key_version: "encryptionkey@0"
I20260924 16:07:04.907068 17531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:04.919425 17531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:04.922871 17531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:04.923086 17531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:04.923692 17531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:04.924603 17531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:04.924681 17531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:04.924772 17531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:04.924824 17531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:04.934016 17531 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:41581
I20260924 16:07:04.934027 17658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:41581 every 8 connection(s)
I20260924 16:07:04.934703 17531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:04.937754 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17531
I20260924 16:07:04.937904 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:04.938289 13345 external_mini_cluster.cc:1428] Setting key a280a7fbf89148550838f7d9568a5cec
Sep 24 16:07:04 dist-test-slave-vj6n krb5kdc[17425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:04.950253 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:04.936510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:39773 (local address 127.13.8.126:40007)
0924 16:07:04.936655 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:04.936658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:04.937455 (+ 797us) server_negotiation.cc:407] Connection header received
0924 16:07:04.939117 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:04.939120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:04.939176 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:04.939288 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:04.940806 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.941501 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.943296 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.943526 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.943607 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:04.946562 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:04.946566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:04.946569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:04.946601 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:04.948263 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:04.948775 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:04.948779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:04.948781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:04.948872 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:04.949186 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:04.949188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:04.949190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:04.949364 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:04.949448 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:04.949941 (+ 493us) server_negotiation.cc:299] Negotiation successful
0924 16:07:04.950060 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:04.951156 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:04.936816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40007 (local address 127.13.8.65:39773)
0924 16:07:04.937294 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:04.937337 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:04.938898 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:04.939472 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:04.939479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:04.939909 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:04.940636 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:04.940661 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.941640 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.941647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:04.943166 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:04.943182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:04.943683 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:04.943690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:04.943934 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:04.944570 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:04.944592 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:04.946389 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:04.948371 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:04.948376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:04.948389 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:04.948633 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:04.948964 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:04.948966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:04.948968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:04.949096 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:04.949458 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:04.949463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:04.949698 (+ 235us) client_negotiation.cc:769] Sending connection context
0924 16:07:04.949882 (+ 184us) client_negotiation.cc:240] Negotiation successful
0924 16:07:04.950201 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:04.952725 17659 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40007
I20260924 16:07:04.953001 17659 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:04.953490 17659 heartbeater.cc:511] Master 127.13.8.126:40007 requested a full tablet report, sending...
I20260924 16:07:04.955139 17469 ts_manager.cc:194] Registered new tserver with Master: a2895615829442b997615d1d0f3cc11e (127.13.8.65:41581)
I20260924 16:07:04.956791 17469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:39773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:04.969928 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:05.080598 17664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:05.080858 17664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:05.080920 17664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:05.084828 17664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:05.084875 17664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:05.084933 17664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:05.090198 17664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:05.091279 17664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:05.092257 17664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:05.101944 17670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:05.101935 17672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.101989 17664 server_base.cc:1034] running on GCE node
W20260924 16:07:05.101934 17669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.102608 17664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:05.103214 17664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:05.104387 17664 hybrid_clock.cc:648] HybridClock initialized: now 1790266025104370 us; error 26 us; skew 500 ppm
Sep 24 16:07:05 dist-test-slave-vj6n krb5kdc[17425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:05.107172 17664 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:05.107883 17664 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:05.107976 17664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:05.108217 17664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:05.110385 17664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "2da864aa7bd94e7c8accb3bf50dcb751"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "d2e9a1a9dfad9c804137f372d4c94136"
server_key_iv: "1493ad5a2cc1033dc7848b877f0e3ee9"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.110944 17664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "2da864aa7bd94e7c8accb3bf50dcb751"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "d2e9a1a9dfad9c804137f372d4c94136"
server_key_iv: "1493ad5a2cc1033dc7848b877f0e3ee9"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.114739 17664 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:07:05.117122 17678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.118345 17664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:05.118468 17664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "2da864aa7bd94e7c8accb3bf50dcb751"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "d2e9a1a9dfad9c804137f372d4c94136"
server_key_iv: "1493ad5a2cc1033dc7848b877f0e3ee9"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.118577 17664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:05.135972 17664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:05.138899 17664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:05.139110 17664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:05.139724 17664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:05.140580 17664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:05.140655 17664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.140725 17664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:05.140776 17664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.150321 17664 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38839
I20260924 16:07:05.150343 17791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38839 every 8 connection(s)
I20260924 16:07:05.150985 17664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:05.156627 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17664
I20260924 16:07:05.156777 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:05.157071 13345 external_mini_cluster.cc:1428] Setting key f8c38b83f587b6aa6b1dd958fee36b1c
Sep 24 16:07:05 dist-test-slave-vj6n krb5kdc[17425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:05.166119 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.152815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:36925 (local address 127.13.8.126:40007)
0924 16:07:05.152956 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.152961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.153905 (+ 944us) server_negotiation.cc:407] Connection header received
0924 16:07:05.155250 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.155253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.155299 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.155372 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:05.156568 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.157340 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.158731 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.159117 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.159254 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.161970 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:05.161976 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:05.161983 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:05.162016 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:05.163913 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.164398 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.164403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.164405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.164502 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.164831 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.164834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.164837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.165090 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:05.165193 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.165739 (+ 546us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.165880 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:05.167024 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.153294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40007 (local address 127.13.8.66:36925)
0924 16:07:05.153746 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.153790 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.155047 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.155579 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.155586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.155924 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:05.156424 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.156438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.157483 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.157487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.158612 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.158621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.159267 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.159270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.159410 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.160080 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:05.160097 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:05.161788 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:05.164028 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:05.164033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:05.164046 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:05.164296 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:05.164603 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:05.164605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:05.164606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:05.164738 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:05.165194 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:05.165200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:05.165447 (+ 247us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.165638 (+ 191us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.165956 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
I20260924 16:07:05.168383 17792 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40007
I20260924 16:07:05.168658 17792 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:05.169164 17792 heartbeater.cc:511] Master 127.13.8.126:40007 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:07:05.170393 17469 ts_manager.cc:194] Registered new tserver with Master: 2da864aa7bd94e7c8accb3bf50dcb751 (127.13.8.66:38839)
I20260924 16:07:05.171067 17469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:36925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:05.184901 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:05.297392 17797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:05.297654 17797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:05.297734 17797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:05.302099 17797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:05.302155 17797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:05.302276 17797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:05.307647 17797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:05.308637 17797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:05.309700 17797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:05.319273 17805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:05.319307 17803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:05.319278 17802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.319703 17797 server_base.cc:1034] running on GCE node
I20260924 16:07:05.320188 17797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:05.320842 17797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:05.322027 17797 hybrid_clock.cc:648] HybridClock initialized: now 1790266025321994 us; error 45 us; skew 500 ppm
Sep 24 16:07:05 dist-test-slave-vj6n krb5kdc[17425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:05.324934 17797 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:05.325672 17797 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:05.325766 17797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:05.325997 17797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:05.328078 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "079a5f8232284e02a7b59e347af1d15c"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "c090ad3b7b403c9a05158026b9122dac"
server_key_iv: "4c80b765ffe5eb65f6e2d87e3401cec8"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.328629 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "079a5f8232284e02a7b59e347af1d15c"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "c090ad3b7b403c9a05158026b9122dac"
server_key_iv: "4c80b765ffe5eb65f6e2d87e3401cec8"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.332212 17797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:07:05.334546 17811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.335692 17797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:05.335820 17797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "079a5f8232284e02a7b59e347af1d15c"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "c090ad3b7b403c9a05158026b9122dac"
server_key_iv: "4c80b765ffe5eb65f6e2d87e3401cec8"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.335925 17797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:05.349665 17797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:05.352975 17797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:05.353166 17797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:05.353770 17797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:05.354668 17797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:05.354748 17797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.354820 17797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:05.354856 17797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.363569 17797 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43899
I20260924 16:07:05.363579 17924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43899 every 8 connection(s)
I20260924 16:07:05.364159 17797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:05.370872 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17797
I20260924 16:07:05.370952 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:05.371273 13345 external_mini_cluster.cc:1428] Setting key eaba8711516a16b02f3faa0c93380786
Sep 24 16:07:05 dist-test-slave-vj6n krb5kdc[17425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:05.377625 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.365921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:44091 (local address 127.13.8.126:40007)
0924 16:07:05.366069 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.366072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.366962 (+ 890us) server_negotiation.cc:407] Connection header received
0924 16:07:05.368282 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.368285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.368343 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.368423 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:05.369558 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.370082 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.371372 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.371651 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.371776 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.374447 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:05.374451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:05.374453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:05.374477 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:05.375629 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.376142 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.376146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.376147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.376219 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.376572 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.376575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.376577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.376760 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:05.376866 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.377352 (+ 486us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.377467 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 16:07:05.378484 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.366358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40007 (local address 127.13.8.67:44091)
0924 16:07:05.366799 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.366849 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.368066 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.368599 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.368605 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.368951 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:05.369406 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.369419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.370208 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.370212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.371223 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.371237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.371850 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.371854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.372067 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.372742 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:05.372768 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:05.374277 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:05.375747 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:05.375754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:05.375769 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:05.376025 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:05.376321 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:05.376325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:05.376327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:05.376467 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:05.376864 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:05.376871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:05.377118 (+ 247us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.377298 (+ 180us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.377627 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":102,"threads_started":1}
I20260924 16:07:05.379518 17925 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40007
I20260924 16:07:05.379758 17925 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:05.380190 17925 heartbeater.cc:511] Master 127.13.8.126:40007 requested a full tablet report, sending...
I20260924 16:07:05.381151 17469 ts_manager.cc:194] Registered new tserver with Master: 079a5f8232284e02a7b59e347af1d15c (127.13.8.67:43899)
I20260924 16:07:05.381696 17469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:44091
I20260924 16:07:05.384948 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:07:05.394627 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.386778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.13.8.126:40007)
0924 16:07:05.386940 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.386945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.387130 (+ 185us) server_negotiation.cc:407] Connection header received
0924 16:07:05.387266 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.387269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.387313 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.387394 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:05.388056 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.388553 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.389811 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.390128 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.390258 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.391604 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:05.391608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:05.391611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:05.391635 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:05.392945 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.393357 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.393361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.393363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.393446 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:05.393719 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:05.393730 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:05.393733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:05.393945 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:05.394085 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.394366 (+ 281us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.394474 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 16:07:05.397347 17469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:05.400445 17469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:05.415315 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.409617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.13.8.66:38839)
0924 16:07:05.410242 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.410289 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.410352 (+ 63us) server_negotiation.cc:407] Connection header received
0924 16:07:05.410436 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.410442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.410706 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.410892 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:05.411246 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412249 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.413709 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.414365 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.414487 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.414663 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.415092 (+ 429us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.415185 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":132,"threads_started":1}
I20260924 16:07:05.415304 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.409148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38839 (local address 127.0.0.1:33852)
0924 16:07:05.410108 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.410122 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.410244 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.410933 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.410940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.410965 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:05.411111 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.411118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412374 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.412380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.413579 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.413588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.414736 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.414739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.414837 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.414896 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.415015 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.415145 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260924 16:07:05.417318 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.408928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43899 (local address 127.0.0.1:41636)
0924 16:07:05.409739 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.409789 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.409950 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.411145 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.411148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.411165 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:05.411286 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.411295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412857 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.412861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.414310 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.414323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.416904 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.416907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.417001 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.417027 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.417112 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.417185 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":146,"thread_start_us":76,"threads_started":1}
I20260924 16:07:05.417362 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.409394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.13.8.67:43899)
0924 16:07:05.410486 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.410498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.410536 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:07:05.410601 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.410606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.410748 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.410981 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:05.411426 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412661 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.415897 (+ 3236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.416798 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.416892 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.416991 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.417141 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.417249 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":220,"threads_started":1}
I20260924 16:07:05.419044 17726 tablet_service.cc:1467] Processing CreateTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 (DEFAULT_TABLE table=test-table [id=d664ba675cf840878b84f16aaf6f2662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:05.420166 17726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1933d6d6fc1427fa7e890bc26396fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:05.419934 17859 tablet_service.cc:1467] Processing CreateTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 (DEFAULT_TABLE table=test-table [id=d664ba675cf840878b84f16aaf6f2662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:05.420634 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.409319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41581 (local address 127.0.0.1:53036)
0924 16:07:05.410468 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.410482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.410597 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.411426 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.411428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.411445 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:05.411516 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.411521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412858 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.412861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.414310 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.414323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.420240 (+ 5917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.420244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.420330 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.420359 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.420437 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.420512 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":118,"thread_start_us":87,"threads_started":1}
I20260924 16:07:05.421015 17859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1933d6d6fc1427fa7e890bc26396fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:05.421084 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.409726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.13.8.65:41581)
0924 16:07:05.410288 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.410295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.410706 (+ 411us) server_negotiation.cc:407] Connection header received
0924 16:07:05.410837 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.410846 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.411122 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.411286 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:05.411649 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.412719 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.414448 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.415302 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.415416 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.415816 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.420805 (+ 4989us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.420907 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":141,"threads_started":1}
I20260924 16:07:05.424088 17593 tablet_service.cc:1467] Processing CreateTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 (DEFAULT_TABLE table=test-table [id=d664ba675cf840878b84f16aaf6f2662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:05.425163 17593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1933d6d6fc1427fa7e890bc26396fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:05.427387 17942 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Bootstrap starting.
I20260924 16:07:05.429018 17943 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Bootstrap starting.
I20260924 16:07:05.429817 17942 tablet_bootstrap.cc:654] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:05.430794 17942 log.cc:824] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:05.431351 17943 tablet_bootstrap.cc:654] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:05.432514 17943 log.cc:824] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:05.432829 17942 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: No bootstrap required, opened a new log
I20260924 16:07:05.433012 17942 ts_tablet_manager.cc:1397] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:07:05.434293 17943 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: No bootstrap required, opened a new log
I20260924 16:07:05.434484 17943 ts_tablet_manager.cc:1397] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:07:05.434610 17945 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: Bootstrap starting.
I20260924 16:07:05.436820 17945 tablet_bootstrap.cc:654] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:05.436627 17942 raft_consensus.cc:348] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.436925 17942 raft_consensus.cc:374] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:05.436986 17942 raft_consensus.cc:729] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da864aa7bd94e7c8accb3bf50dcb751, State: Initialized, Role: FOLLOWER
I20260924 16:07:05.437538 17942 consensus_queue.cc:260] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.437472 17943 raft_consensus.cc:348] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.437736 17943 raft_consensus.cc:374] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:05.437789 17943 raft_consensus.cc:729] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079a5f8232284e02a7b59e347af1d15c, State: Initialized, Role: FOLLOWER
I20260924 16:07:05.438058 17945 log.cc:824] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:05.438247 17943 consensus_queue.cc:260] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.438349 17792 heartbeater.cc:503] Master 127.13.8.126:40007 was elected leader, sending a full tablet report...
I20260924 16:07:05.438627 17942 ts_tablet_manager.cc:1428] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:07:05.439221 17943 ts_tablet_manager.cc:1428] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260924 16:07:05.439606 17925 heartbeater.cc:503] Master 127.13.8.126:40007 was elected leader, sending a full tablet report...
I20260924 16:07:05.440809 17945 tablet_bootstrap.cc:492] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: No bootstrap required, opened a new log
I20260924 16:07:05.441006 17945 ts_tablet_manager.cc:1397] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260924 16:07:05.444425 17945 raft_consensus.cc:348] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.444653 17945 raft_consensus.cc:374] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:05.444742 17945 raft_consensus.cc:729] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2895615829442b997615d1d0f3cc11e, State: Initialized, Role: FOLLOWER
I20260924 16:07:05.445158 17945 consensus_queue.cc:260] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.445858 17659 heartbeater.cc:503] Master 127.13.8.126:40007 was elected leader, sending a full tablet report...
I20260924 16:07:05.446177 17945 ts_tablet_manager.cc:1428] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260924 16:07:05.482667 17947 raft_consensus.cc:482] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:05.482926 17947 raft_consensus.cc:504] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.484136 17947 leader_election.cc:288] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 079a5f8232284e02a7b59e347af1d15c (127.13.8.67:43899), a2895615829442b997615d1d0f3cc11e (127.13.8.65:41581)
I20260924 16:07:05.488286 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.484438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41581 (local address 127.13.8.66:49229)
0924 16:07:05.484598 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.484615 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.484722 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.485040 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.485044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.485074 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:05.485195 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.485202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.486150 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.486154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.487195 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.487205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.487862 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.487973 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.488002 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.488071 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.488145 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"spinlock_wait_cycles":768}
I20260924 16:07:05.488337 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.484544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:49229 (local address 127.13.8.65:41581)
0924 16:07:05.484667 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.484672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.484691 (+ 19us) server_negotiation.cc:407] Connection header received
0924 16:07:05.484829 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.484832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.484885 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.484977 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:05.485307 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.486033 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487323 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.487742 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487820 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.487895 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.488107 (+ 212us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.488197 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:07:05.488385 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.484554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53671 (local address 127.13.8.67:43899)
0924 16:07:05.484680 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.484686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.485068 (+ 382us) server_negotiation.cc:407] Connection header received
0924 16:07:05.485156 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.485161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.485209 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.485280 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:05.485528 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.486172 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487396 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.487811 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487914 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.487976 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.488188 (+ 212us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.488259 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:07:05.488801 17951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.484440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43899 (local address 127.13.8.66:53671)
0924 16:07:05.484945 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.484960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:05.485062 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:05.485304 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:05.485307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:05.485324 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:05.485427 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.485432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.486286 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.486289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.487244 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:05.487254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.487936 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.487939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:05.488039 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.488082 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 16:07:05.488157 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 16:07:05.488216 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":127,"thread_start_us":138,"threads_started":1}
I20260924 16:07:05.489080 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1933d6d6fc1427fa7e890bc26396fb9" candidate_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2895615829442b997615d1d0f3cc11e" is_pre_election: true
I20260924 16:07:05.489414 17613 raft_consensus.cc:2454] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da864aa7bd94e7c8accb3bf50dcb751 in term 0.
I20260924 16:07:05.489427 17879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1933d6d6fc1427fa7e890bc26396fb9" candidate_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079a5f8232284e02a7b59e347af1d15c" is_pre_election: true
I20260924 16:07:05.489777 17879 raft_consensus.cc:2454] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da864aa7bd94e7c8accb3bf50dcb751 in term 0.
I20260924 16:07:05.489909 17679 leader_election.cc:302] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2da864aa7bd94e7c8accb3bf50dcb751, a2895615829442b997615d1d0f3cc11e; no voters:
I20260924 16:07:05.490236 17947 raft_consensus.cc:2781] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:05.490329 17947 raft_consensus.cc:482] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:05.490386 17947 raft_consensus.cc:3037] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:05.491382 17947 raft_consensus.cc:504] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.491716 17947 leader_election.cc:288] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [CANDIDATE]: Term 1 election: Requested vote from peers 079a5f8232284e02a7b59e347af1d15c (127.13.8.67:43899), a2895615829442b997615d1d0f3cc11e (127.13.8.65:41581)
I20260924 16:07:05.492012 17879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1933d6d6fc1427fa7e890bc26396fb9" candidate_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079a5f8232284e02a7b59e347af1d15c"
I20260924 16:07:05.492110 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1933d6d6fc1427fa7e890bc26396fb9" candidate_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2895615829442b997615d1d0f3cc11e"
I20260924 16:07:05.492116 17879 raft_consensus.cc:3037] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:05.492288 17613 raft_consensus.cc:3037] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:05.493144 17879 raft_consensus.cc:2454] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da864aa7bd94e7c8accb3bf50dcb751 in term 1.
I20260924 16:07:05.493345 17613 raft_consensus.cc:2454] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da864aa7bd94e7c8accb3bf50dcb751 in term 1.
I20260924 16:07:05.493489 17681 leader_election.cc:302] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 079a5f8232284e02a7b59e347af1d15c, 2da864aa7bd94e7c8accb3bf50dcb751; no voters:
I20260924 16:07:05.493745 17947 raft_consensus.cc:2781] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:05.493937 17947 raft_consensus.cc:686] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 1 LEADER]: Becoming Leader. State: Replica: 2da864aa7bd94e7c8accb3bf50dcb751, State: Running, Role: LEADER
I20260924 16:07:05.494298 17947 consensus_queue.cc:237] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } }
I20260924 16:07:05.497537 17469 catalog_manager.cc:5495] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2da864aa7bd94e7c8accb3bf50dcb751 (127.13.8.66). New cstate: current_term: 1 leader_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2da864aa7bd94e7c8accb3bf50dcb751" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:05.513833 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.510010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.13.8.66:38839)
0924 16:07:05.510138 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.510143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.510197 (+ 54us) server_negotiation.cc:407] Connection header received
0924 16:07:05.510420 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.510422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.510493 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.510626 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:05.510890 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.511464 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.512344 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:05.512570 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:05.512675 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:05.512996 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:05.513422 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:05.513557 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.513623 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.513679 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:07:05.519938 17879 raft_consensus.cc:1261] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 1 FOLLOWER]: Refusing update from remote peer 2da864aa7bd94e7c8accb3bf50dcb751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:05.519938 17613 raft_consensus.cc:1261] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 1 FOLLOWER]: Refusing update from remote peer 2da864aa7bd94e7c8accb3bf50dcb751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:05.520740 17952 consensus_queue.cc:1035] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "079a5f8232284e02a7b59e347af1d15c" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:05.520964 17947 consensus_queue.cc:1035] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2895615829442b997615d1d0f3cc11e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:05.528581 17956 mvcc.cc:205] Tried to move back new op lower bound from 7332929640521060352 to 7332929640427307008. Current Snapshot: MvccSnapshot[applied={T|T < 7332929640521060352}]
I20260924 16:07:05.530354 17957 mvcc.cc:205] Tried to move back new op lower bound from 7332929640521060352 to 7332929640427307008. Current Snapshot: MvccSnapshot[applied={T|T < 7332929640521060352}]
I20260924 16:07:05.537952 17469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:05.538151 17469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:05.540455 17469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5c9822e3ac347feb48c91253237d195: Sending DeleteTablet for 3 replicas of tablet c1933d6d6fc1427fa7e890bc26396fb9
I20260924 16:07:05.541203 17726 tablet_service.cc:1514] Processing DeleteTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:05 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:33852
I20260924 16:07:05.541209 17859 tablet_service.cc:1514] Processing DeleteTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:05 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41636
I20260924 16:07:05.541320 17593 tablet_service.cc:1514] Processing DeleteTablet for tablet c1933d6d6fc1427fa7e890bc26396fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:05 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:53036
I20260924 16:07:05.541741 17967 tablet_replica.cc:330] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e: stopping tablet replica
I20260924 16:07:05.542079 17967 raft_consensus.cc:2229] T c1933d6d6fc1427fa7e890bc26396fb9 P a2895615829442b997615d1d0f3cc11e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:05.542368 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17531
I20260924 16:07:05.543140 17966 tablet_replica.cc:330] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: stopping tablet replica
I20260924 16:07:05.543442 17966 raft_consensus.cc:2229] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:05.543756 17966 raft_consensus.cc:2258] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:05.545480 17966 ts_tablet_manager.cc:1905] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:05.548045 17965 tablet_replica.cc:330] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: stopping tablet replica
I20260924 16:07:05.548329 17965 raft_consensus.cc:2229] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:05.548741 17966 ts_tablet_manager.cc:1918] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:05.548772 17965 raft_consensus.cc:2258] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:05.548831 17966 log.cc:1196] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/c1933d6d6fc1427fa7e890bc26396fb9
I20260924 16:07:05.549185 17966 ts_tablet_manager.cc:1939] T c1933d6d6fc1427fa7e890bc26396fb9 P 079a5f8232284e02a7b59e347af1d15c: Deleting consensus metadata
I20260924 16:07:05.550235 17965 ts_tablet_manager.cc:1905] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:05.550343 17455 catalog_manager.cc:4841] TS 079a5f8232284e02a7b59e347af1d15c (127.13.8.67:43899): tablet c1933d6d6fc1427fa7e890bc26396fb9 (table test-table [id=d664ba675cf840878b84f16aaf6f2662]) successfully deleted
I20260924 16:07:05.552532 17965 ts_tablet_manager.cc:1918] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:05.552646 17965 log.cc:1196] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/c1933d6d6fc1427fa7e890bc26396fb9
I20260924 16:07:05.552937 17965 ts_tablet_manager.cc:1939] T c1933d6d6fc1427fa7e890bc26396fb9 P 2da864aa7bd94e7c8accb3bf50dcb751: Deleting consensus metadata
W20260924 16:07:05.553241 17453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:41581 (error 108)
I20260924 16:07:05.554081 17454 catalog_manager.cc:4841] TS 2da864aa7bd94e7c8accb3bf50dcb751 (127.13.8.66:38839): tablet c1933d6d6fc1427fa7e890bc26396fb9 (table test-table [id=d664ba675cf840878b84f16aaf6f2662]) successfully deleted
I20260924 16:07:05.554208 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.553923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41581 (local address 127.0.0.1:53046)
0924 16:07:05.554033 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:05.554107 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 16:07:05.554296 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17664
W20260924 16:07:05.554402 17453 catalog_manager.cc:4584] TS a2895615829442b997615d1d0f3cc11e (127.13.8.65:41581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1933d6d6fc1427fa7e890bc26396fb9: Network error: Client connection negotiation failed: client connection to 127.13.8.65:41581: connect: Connection refused (error 111)
I20260924 16:07:05.560384 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17797
I20260924 16:07:05.566039 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17438
2026-09-24T16:07:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2325 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T16:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:05Z Disabled control of system clock
I20260924 16:07:05.596913 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40155
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 16:07:05.707346 17972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:05.707592 17972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:05.707628 17972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:05.711668 17972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:05.711716 17972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:05.711731 17972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:05.711747 17972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:05.711762 17972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:05.716634 17972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:40155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:05.717841 17972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:05.719390 17972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:05.727906 17977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.728065 17972 server_base.cc:1034] running on GCE node
W20260924 16:07:05.727957 17980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:05.727977 17978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.728737 17972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:05.729835 17972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:05.730993 17972 hybrid_clock.cc:648] HybridClock initialized: now 1790266025730959 us; error 44 us; skew 500 ppm
I20260924 16:07:05.733117 17972 webserver.cc:460] Webserver started at http://127.13.8.126:34771/ using document root <none> and password file <none>
I20260924 16:07:05.733750 17972 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:05.733850 17972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:05.734067 17972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:05.736224 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "e6d50685aef14eada7c5bb62f02dd4d1"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "1bcf1f103b57ac262d423e3a46d1e1f1"
server_key_iv: "b6e9448ea26dc8ffb36bf9d107667116"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.736799 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "e6d50685aef14eada7c5bb62f02dd4d1"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "1bcf1f103b57ac262d423e3a46d1e1f1"
server_key_iv: "b6e9448ea26dc8ffb36bf9d107667116"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.740502 17972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:07:05.743048 17986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.744306 17972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:05.744573 17972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "e6d50685aef14eada7c5bb62f02dd4d1"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "1bcf1f103b57ac262d423e3a46d1e1f1"
server_key_iv: "b6e9448ea26dc8ffb36bf9d107667116"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.744695 17972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:05.749444 17972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:05.750126 17972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:05.750339 17972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:05.757563 17972 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:40155
I20260924 16:07:05.757558 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:40155 every 8 connection(s)
I20260924 16:07:05.758646 17972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:05.762411 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 17972
I20260924 16:07:05.762508 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:05.762739 13345 external_mini_cluster.cc:1428] Setting key 31e5353a117d860c076814106cfbcbdb
I20260924 16:07:05.763434 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:05.768326 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:05.764074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.13.8.126:40155)
0924 16:07:05.764761 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:05.764774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:05.764809 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:07:05.765445 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:05.765462 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:05.765817 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:05.766124 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:05.767005 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:05.767010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:05.767022 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:05.767031 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:05.767085 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:05.767101 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:05.767216 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:05.767333 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 16:07:05.767613 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":140,"threads_started":1}
I20260924 16:07:05.770493 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: Bootstrap starting.
I20260924 16:07:05.772768 18039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:05.773505 18039 log.cc:824] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:05.775650 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: No bootstrap required, opened a new log
I20260924 16:07:05.779254 18039 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } }
I20260924 16:07:05.779469 18039 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:05.779556 18039 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d50685aef14eada7c5bb62f02dd4d1, State: Initialized, Role: FOLLOWER
I20260924 16:07:05.779963 18039 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } }
I20260924 16:07:05.780107 18039 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:05.780169 18039 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:05.780256 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:05.781289 18039 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } }
I20260924 16:07:05.781635 18039 leader_election.cc:302] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6d50685aef14eada7c5bb62f02dd4d1; no voters:
I20260924 16:07:05.782019 18039 leader_election.cc:288] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:05.782078 18044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:05.782321 18044 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [term 1 LEADER]: Becoming Leader. State: Replica: e6d50685aef14eada7c5bb62f02dd4d1, State: Running, Role: LEADER
I20260924 16:07:05.782615 18044 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } }
I20260924 16:07:05.783375 18039 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:05.784350 18046 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } } }
I20260924 16:07:05.784389 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6d50685aef14eada7c5bb62f02dd4d1. Latest consensus state: current_term: 1 leader_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d50685aef14eada7c5bb62f02dd4d1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40155 } } }
I20260924 16:07:05.784479 18046 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:05.784507 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:05.784950 18053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:05.788224 18053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:05.793684 18053 catalog_manager.cc:1347] Generated new cluster ID: 5bc3ef88d959434fa15c90cc8d4bc932
I20260924 16:07:05.793751 18053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:05.825978 18053 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:05.826392 18053 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:05.833930 18053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: Generated new TSK 0
I20260924 16:07:05.834753 18053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:05.844024 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--builtin_ntp_servers=127.13.8.84:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 16:07:05.958139 18063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:05.958447 18063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:05.958485 18063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:05.964054 18063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:05.964110 18063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:05.964206 18063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:05.968951 18063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:05.970016 18063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:05.971554 18063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:05.981859 18068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:05.981966 18071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.982048 18063 server_base.cc:1034] running on GCE node
W20260924 16:07:05.981896 18069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:05.982576 18063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:05.983309 18063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:05.984505 18063 hybrid_clock.cc:648] HybridClock initialized: now 1790266025984465 us; error 48 us; skew 500 ppm
I20260924 16:07:05.986542 18063 webserver.cc:460] Webserver started at http://127.13.8.65:39995/ using document root <none> and password file <none>
I20260924 16:07:05.987110 18063 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:05.987169 18063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:05.987336 18063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:05.989396 18063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "7568378ef2a64afa8632ea4e88f27a0a"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "4d773aa073367c759c8baaaded164f5e"
server_key_iv: "e81dd0ed4f9803e1fb014fc5c13e8a4b"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.989997 18063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "7568378ef2a64afa8632ea4e88f27a0a"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "4d773aa073367c759c8baaaded164f5e"
server_key_iv: "e81dd0ed4f9803e1fb014fc5c13e8a4b"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.993952 18063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260924 16:07:05.997071 18077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:05.998564 18063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:05.998677 18063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "7568378ef2a64afa8632ea4e88f27a0a"
format_stamp: "Formatted at 2026-09-24 16:07:05 on dist-test-slave-vj6n"
server_key: "4d773aa073367c759c8baaaded164f5e"
server_key_iv: "e81dd0ed4f9803e1fb014fc5c13e8a4b"
server_key_version: "encryptionkey@0"
I20260924 16:07:05.998778 18063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:06.003450 18063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:06.004045 18063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:06.004182 18063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:06.004684 18063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:06.005553 18063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:06.005599 18063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.005637 18063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:06.005658 18063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.014724 18063 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43355
I20260924 16:07:06.014802 18190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43355 every 8 connection(s)
I20260924 16:07:06.015735 18063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:06.020458 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18063
I20260924 16:07:06.020607 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:06.020913 13345 external_mini_cluster.cc:1428] Setting key 675d108a591c565fb6a18087c73c6574
I20260924 16:07:06.022176 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.017607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:40309 (local address 127.13.8.126:40155)
0924 16:07:06.017784 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.017789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.018983 (+ 1194us) server_negotiation.cc:407] Connection header received
0924 16:07:06.020258 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.020262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.020309 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.020413 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.021666 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.021669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.021672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.021684 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.021744 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.021752 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.021843 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.022000 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.022044 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 16:07:06.022921 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--builtin_ntp_servers=127.13.8.84:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 16:07:06.023099 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.018108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40155 (local address 127.13.8.65:40309)
0924 16:07:06.018833 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.018875 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.020038 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.020585 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.020595 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.020735 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.021370 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.021372 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.021426 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.021525 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.021843 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.021853 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.021862 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.022045 (+ 183us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.022324 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":127,"threads_started":1}
I20260924 16:07:06.024415 18191 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40155
I20260924 16:07:06.024678 18191 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:06.025095 18191 heartbeater.cc:511] Master 127.13.8.126:40155 requested a full tablet report, sending...
I20260924 16:07:06.026714 18003 ts_manager.cc:194] Registered new tserver with Master: 7568378ef2a64afa8632ea4e88f27a0a (127.13.8.65:43355)
W20260924 16:07:06.139585 18195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:06.139844 18195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:06.139931 18195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:06.144007 18195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:06.144065 18195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:06.144156 18195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:06.149418 18195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:06.150560 18195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:06.151985 18195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:06.161571 18200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:06.161600 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:06.161592 18203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:06.161746 18195 server_base.cc:1034] running on GCE node
I20260924 16:07:06.162395 18195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:06.163009 18195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:06.164196 18195 hybrid_clock.cc:648] HybridClock initialized: now 1790266026164154 us; error 51 us; skew 500 ppm
I20260924 16:07:06.166090 18195 webserver.cc:460] Webserver started at http://127.13.8.66:33403/ using document root <none> and password file <none>
I20260924 16:07:06.166708 18195 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:06.166767 18195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:06.166991 18195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:06.168995 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "8e394b474ae84d0ca2ec53eaf0ca8335"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "da051364cc6c306920591bcc25e93b88"
server_key_iv: "92185bda5fca11b1d87a7c576144ceb2"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.169559 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "8e394b474ae84d0ca2ec53eaf0ca8335"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "da051364cc6c306920591bcc25e93b88"
server_key_iv: "92185bda5fca11b1d87a7c576144ceb2"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.173192 18195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:07:06.175671 18209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.176873 18195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:06.177002 18195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "8e394b474ae84d0ca2ec53eaf0ca8335"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "da051364cc6c306920591bcc25e93b88"
server_key_iv: "92185bda5fca11b1d87a7c576144ceb2"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.177112 18195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:06.181592 18195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:06.182166 18195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:06.182365 18195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:06.182917 18195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:06.183918 18195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:06.183979 18195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.184039 18195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:06.184084 18195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.192786 18195 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:42401
I20260924 16:07:06.192821 18323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:42401 every 8 connection(s)
I20260924 16:07:06.193661 18195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:06.199355 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18195
I20260924 16:07:06.199467 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:06.199616 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.195405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:47129 (local address 127.13.8.126:40155)
0924 16:07:06.195556 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.195561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.196662 (+ 1101us) server_negotiation.cc:407] Connection header received
0924 16:07:06.197929 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.197932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.197967 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.198039 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.199136 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.199139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.199141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.199152 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.199169 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.199176 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.199253 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.199444 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.199490 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:06.199772 13345 external_mini_cluster.cc:1428] Setting key f02f394ee6461a430a7331e60fc311a2
I20260924 16:07:06.200569 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.195862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40155 (local address 127.13.8.66:47129)
0924 16:07:06.196508 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.196554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.197747 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.198201 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.198207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.198344 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.198883 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.198884 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.198923 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.198987 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.199261 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.199270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.199277 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.199463 (+ 186us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.199751 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":99,"threads_started":1}
I20260924 16:07:06.201798 18324 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40155
I20260924 16:07:06.202033 18324 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:06.202008 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--builtin_ntp_servers=127.13.8.84:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 16:07:06.202543 18324 heartbeater.cc:511] Master 127.13.8.126:40155 requested a full tablet report, sending...
I20260924 16:07:06.203641 18003 ts_manager.cc:194] Registered new tserver with Master: 8e394b474ae84d0ca2ec53eaf0ca8335 (127.13.8.66:42401)
W20260924 16:07:06.313812 18327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:06.314066 18327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:06.314105 18327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:06.318089 18327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:06.318142 18327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:06.318259 18327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:06.322932 18327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:40155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:06.324095 18327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:06.325593 18327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:06.335755 18335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:06.335789 18333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:06.335789 18332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:06.336040 18327 server_base.cc:1034] running on GCE node
I20260924 16:07:06.336479 18327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:06.337085 18327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:06.338295 18327 hybrid_clock.cc:648] HybridClock initialized: now 1790266026338255 us; error 56 us; skew 500 ppm
I20260924 16:07:06.340034 18327 webserver.cc:460] Webserver started at http://127.13.8.67:43113/ using document root <none> and password file <none>
I20260924 16:07:06.340618 18327 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:06.340705 18327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:06.341003 18327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:06.343051 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "dd41a8fb09c6413c8265b09a1144ae3e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "7e06bb43ca1568c98a5c95e88cf86de8"
server_key_iv: "3c842fa965f0f4452496948ec1d76aad"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.343596 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "dd41a8fb09c6413c8265b09a1144ae3e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "7e06bb43ca1568c98a5c95e88cf86de8"
server_key_iv: "3c842fa965f0f4452496948ec1d76aad"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.347863 18327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:07:06.350941 18341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.352174 18327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:06.352268 18327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "dd41a8fb09c6413c8265b09a1144ae3e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "7e06bb43ca1568c98a5c95e88cf86de8"
server_key_iv: "3c842fa965f0f4452496948ec1d76aad"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.352366 18327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:06.356801 18327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:06.357350 18327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:06.357483 18327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:06.358098 18327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:06.358996 18327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:06.359046 18327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.359107 18327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:06.359138 18327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.368211 18327 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:35115
I20260924 16:07:06.368238 18455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:35115 every 8 connection(s)
I20260924 16:07:06.369088 18327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:06.375348 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.370998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:57869 (local address 127.13.8.126:40155)
0924 16:07:06.371157 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.371162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.372313 (+ 1151us) server_negotiation.cc:407] Connection header received
0924 16:07:06.373711 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.373713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.373758 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.373832 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.375000 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.375003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.375005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.375015 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.375033 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.375041 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.375113 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.375206 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.375244 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 16:07:06.376194 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.371455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40155 (local address 127.13.8.67:57869)
0924 16:07:06.372168 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.372210 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.373514 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.374000 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.374007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.374108 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.374754 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.374756 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.374800 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.374870 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.375098 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.375102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.375105 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.375227 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.375416 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":117,"threads_started":1}
I20260924 16:07:06.377291 18456 heartbeater.cc:347] Connected to a master server at 127.13.8.126:40155
I20260924 16:07:06.377539 18456 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:06.378000 18456 heartbeater.cc:511] Master 127.13.8.126:40155 requested a full tablet report, sending...
I20260924 16:07:06.378150 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18327
I20260924 16:07:06.378291 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:06.378608 13345 external_mini_cluster.cc:1428] Setting key 542c9169e03f42e3a076bfc2a6d247c2
I20260924 16:07:06.379228 18003 ts_manager.cc:194] Registered new tserver with Master: dd41a8fb09c6413c8265b09a1144ae3e (127.13.8.67:35115)
I20260924 16:07:06.380704 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:07:06.384436 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.382794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.13.8.126:40155)
0924 16:07:06.382963 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.382967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.383109 (+ 142us) server_negotiation.cc:407] Connection header received
0924 16:07:06.383223 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.383225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.383256 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.383317 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.384047 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.384053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.384055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.384067 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.384088 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.384097 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.384173 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.384235 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.384265 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 16:07:06.387292 18003 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:06.390201 18003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:06.401643 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.398317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43355 (local address 127.0.0.1:49252)
0924 16:07:06.398710 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.398775 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.398856 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.399819 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.399826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.399903 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.400631 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.400633 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.400649 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.400703 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.401273 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.401284 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.401290 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.401415 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.401489 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":73,"threads_started":1}
I20260924 16:07:06.401698 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.398571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.13.8.65:43355)
0924 16:07:06.399339 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.399352 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.399378 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:07:06.399433 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.399441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.399637 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.400402 (+ 765us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.401037 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.401042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.401064 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.401080 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.401141 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.401173 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.401291 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.401429 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.401509 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":108,"threads_started":1}
I20260924 16:07:06.402055 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.399289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.13.8.67:35115)
0924 16:07:06.399735 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.399749 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.400000 (+ 251us) server_negotiation.cc:407] Connection header received
0924 16:07:06.400120 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.400127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.400341 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.400460 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.401389 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.401396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.401408 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.401420 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.401463 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.401497 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.401593 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.401796 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.401873 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":154,"threads_started":1}
I20260924 16:07:06.402268 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.398314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42401 (local address 127.0.0.1:46398)
0924 16:07:06.399301 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.399314 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.399399 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.400144 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.400147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.400160 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.401052 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.401055 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.401066 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.401095 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.401498 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.401501 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.401504 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.401570 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.401596 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":527,"thread_start_us":91,"threads_started":1}
I20260924 16:07:06.402570 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.398459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.13.8.66:42401)
0924 16:07:06.399130 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.399148 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.399489 (+ 341us) server_negotiation.cc:407] Connection header received
0924 16:07:06.399541 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.399545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.399679 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.399812 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.401241 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.401251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.401265 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.401278 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.401323 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.401364 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.401461 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.402343 (+ 882us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.402418 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":113,"threads_started":1}
I20260924 16:07:06.402779 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.399103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:35115 (local address 127.0.0.1:37304)
0924 16:07:06.399884 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.399905 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.400009 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.400818 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.400820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.400831 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.401235 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.401240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.401252 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.401278 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.401636 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.401640 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.401643 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.402611 (+ 968us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.402652 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":95,"threads_started":1}
I20260924 16:07:06.404922 18125 tablet_service.cc:1467] Processing CreateTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 (DEFAULT_TABLE table=test-table [id=f63c04d7f71d46628558d7f007ebe5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:06.405377 18389 tablet_service.cc:1467] Processing CreateTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 (DEFAULT_TABLE table=test-table [id=f63c04d7f71d46628558d7f007ebe5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:06.405377 18257 tablet_service.cc:1467] Processing CreateTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 (DEFAULT_TABLE table=test-table [id=f63c04d7f71d46628558d7f007ebe5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:06.406059 18125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532e07d68e2a42a7aa19dfc101cfaad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:06.406431 18257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532e07d68e2a42a7aa19dfc101cfaad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:06.406431 18389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532e07d68e2a42a7aa19dfc101cfaad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:06.413199 18473 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Bootstrap starting.
I20260924 16:07:06.413915 18475 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Bootstrap starting.
I20260924 16:07:06.415582 18473 tablet_bootstrap.cc:654] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:06.415834 18475 tablet_bootstrap.cc:654] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:06.416392 18473 log.cc:824] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:06.416558 18475 log.cc:824] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:06.417150 18474 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Bootstrap starting.
I20260924 16:07:06.418356 18475 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: No bootstrap required, opened a new log
I20260924 16:07:06.418529 18473 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: No bootstrap required, opened a new log
I20260924 16:07:06.418565 18475 ts_tablet_manager.cc:1397] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:07:06.418712 18473 ts_tablet_manager.cc:1397] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:07:06.419309 18474 tablet_bootstrap.cc:654] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:06.420142 18474 log.cc:824] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:06.422335 18474 tablet_bootstrap.cc:492] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: No bootstrap required, opened a new log
I20260924 16:07:06.422525 18474 ts_tablet_manager.cc:1397] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260924 16:07:06.422879 18475 raft_consensus.cc:348] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.423131 18475 raft_consensus.cc:374] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:06.423184 18475 raft_consensus.cc:729] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e394b474ae84d0ca2ec53eaf0ca8335, State: Initialized, Role: FOLLOWER
I20260924 16:07:06.423135 18473 raft_consensus.cc:348] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.423393 18473 raft_consensus.cc:374] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:06.423463 18473 raft_consensus.cc:729] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7568378ef2a64afa8632ea4e88f27a0a, State: Initialized, Role: FOLLOWER
I20260924 16:07:06.423692 18475 consensus_queue.cc:260] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.423894 18473 consensus_queue.cc:260] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.424571 18191 heartbeater.cc:503] Master 127.13.8.126:40155 was elected leader, sending a full tablet report...
I20260924 16:07:06.424796 18475 ts_tablet_manager.cc:1428] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:07:06.424829 18473 ts_tablet_manager.cc:1428] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:07:06.425101 18324 heartbeater.cc:503] Master 127.13.8.126:40155 was elected leader, sending a full tablet report...
I20260924 16:07:06.426757 18474 raft_consensus.cc:348] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.427018 18474 raft_consensus.cc:374] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:06.427078 18474 raft_consensus.cc:729] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd41a8fb09c6413c8265b09a1144ae3e, State: Initialized, Role: FOLLOWER
I20260924 16:07:06.427484 18474 consensus_queue.cc:260] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.428141 18456 heartbeater.cc:503] Master 127.13.8.126:40155 was elected leader, sending a full tablet report...
I20260924 16:07:06.429268 18474 ts_tablet_manager.cc:1428] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260924 16:07:06.444639 18325 tablet.cc:2369] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:06.511739 18480 raft_consensus.cc:482] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:06.511982 18480 raft_consensus.cc:504] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.513276 18480 leader_election.cc:288] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e394b474ae84d0ca2ec53eaf0ca8335 (127.13.8.66:42401), dd41a8fb09c6413c8265b09a1144ae3e (127.13.8.67:35115)
I20260924 16:07:06.515239 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.513515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42401 (local address 127.13.8.65:48779)
0924 16:07:06.513674 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.513693 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.513783 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.514064 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.514067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.514082 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.514596 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.514598 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.514608 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.514656 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.514961 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.514964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.514966 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.515069 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.515108 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:07:06.515250 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.513653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:48779 (local address 127.13.8.66:42401)
0924 16:07:06.513803 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.513810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.513830 (+ 20us) server_negotiation.cc:407] Connection header received
0924 16:07:06.513886 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.513889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.513930 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.514048 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.514819 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.514823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.514825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.514837 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.514862 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.514874 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.514967 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.515078 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.515120 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 16:07:06.515430 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.513836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:52635 (local address 127.13.8.67:35115)
0924 16:07:06.513963 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.513968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.514115 (+ 147us) server_negotiation.cc:407] Connection header received
0924 16:07:06.514280 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.514285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.514333 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.514422 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.515048 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.515051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.515053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.515065 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.515093 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.515100 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.515192 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.515277 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.515314 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:07:06.515453 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.513693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:35115 (local address 127.13.8.65:52635)
0924 16:07:06.514020 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.514031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.514133 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.514446 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.514449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.514464 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:06.514895 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:06.514900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:06.514911 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:06.514941 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:06.515187 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:06.515190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:06.515194 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 16:07:06.515273 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 16:07:06.515309 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
I20260924 16:07:06.516064 18277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532e07d68e2a42a7aa19dfc101cfaad0" candidate_uuid: "7568378ef2a64afa8632ea4e88f27a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" is_pre_election: true
I20260924 16:07:06.516064 18409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532e07d68e2a42a7aa19dfc101cfaad0" candidate_uuid: "7568378ef2a64afa8632ea4e88f27a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" is_pre_election: true
I20260924 16:07:06.516381 18277 raft_consensus.cc:2454] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7568378ef2a64afa8632ea4e88f27a0a in term 0.
I20260924 16:07:06.516388 18409 raft_consensus.cc:2454] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7568378ef2a64afa8632ea4e88f27a0a in term 0.
W20260924 16:07:06.516885 18193 tablet.cc:2369] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:06.516906 18079 leader_election.cc:302] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7568378ef2a64afa8632ea4e88f27a0a, 8e394b474ae84d0ca2ec53eaf0ca8335, dd41a8fb09c6413c8265b09a1144ae3e; no voters:
I20260924 16:07:06.517194 18480 raft_consensus.cc:2781] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:06.517278 18480 raft_consensus.cc:482] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:06.517321 18480 raft_consensus.cc:3037] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:06.518440 18480 raft_consensus.cc:504] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.518822 18480 leader_election.cc:288] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [CANDIDATE]: Term 1 election: Requested vote from peers 8e394b474ae84d0ca2ec53eaf0ca8335 (127.13.8.66:42401), dd41a8fb09c6413c8265b09a1144ae3e (127.13.8.67:35115)
I20260924 16:07:06.519124 18277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532e07d68e2a42a7aa19dfc101cfaad0" candidate_uuid: "7568378ef2a64afa8632ea4e88f27a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335"
I20260924 16:07:06.519203 18409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532e07d68e2a42a7aa19dfc101cfaad0" candidate_uuid: "7568378ef2a64afa8632ea4e88f27a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd41a8fb09c6413c8265b09a1144ae3e"
I20260924 16:07:06.519248 18277 raft_consensus.cc:3037] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:06.519358 18409 raft_consensus.cc:3037] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:06.520313 18277 raft_consensus.cc:2454] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7568378ef2a64afa8632ea4e88f27a0a in term 1.
I20260924 16:07:06.520346 18409 raft_consensus.cc:2454] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7568378ef2a64afa8632ea4e88f27a0a in term 1.
I20260924 16:07:06.520687 18080 leader_election.cc:302] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7568378ef2a64afa8632ea4e88f27a0a, dd41a8fb09c6413c8265b09a1144ae3e; no voters:
I20260924 16:07:06.520861 18480 raft_consensus.cc:2781] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:06.521085 18480 raft_consensus.cc:686] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 1 LEADER]: Becoming Leader. State: Replica: 7568378ef2a64afa8632ea4e88f27a0a, State: Running, Role: LEADER
I20260924 16:07:06.521405 18480 consensus_queue.cc:237] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } }
I20260924 16:07:06.524811 18003 catalog_manager.cc:5495] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7568378ef2a64afa8632ea4e88f27a0a (127.13.8.65). New cstate: current_term: 1 leader_uuid: "7568378ef2a64afa8632ea4e88f27a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7568378ef2a64afa8632ea4e88f27a0a" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:06.555153 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.553333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.13.8.65:43355)
0924 16:07:06.553486 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.553490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.553504 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:07:06.553639 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.553641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.553704 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.553786 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.554618 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.554622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.554624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.554637 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.554679 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.554688 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.554784 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.554889 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.554940 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:07:06.561419 18409 raft_consensus.cc:1261] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 1 FOLLOWER]: Refusing update from remote peer 7568378ef2a64afa8632ea4e88f27a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:06.561419 18277 raft_consensus.cc:1261] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 1 FOLLOWER]: Refusing update from remote peer 7568378ef2a64afa8632ea4e88f27a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:06.562192 18480 consensus_queue.cc:1035] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd41a8fb09c6413c8265b09a1144ae3e" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:06.562448 18483 consensus_queue.cc:1035] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e394b474ae84d0ca2ec53eaf0ca8335" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:06.568301 18486 mvcc.cc:205] Tried to move back new op lower bound from 7332929644787650560 to 7332929644634497024. Current Snapshot: MvccSnapshot[applied={T|T < 7332929644787650560}]
I20260924 16:07:06.577914 18003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:06.578082 18003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:06.580657 18003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6d50685aef14eada7c5bb62f02dd4d1: Sending DeleteTablet for 3 replicas of tablet 532e07d68e2a42a7aa19dfc101cfaad0
I20260924 16:07:06.581578 18125 tablet_service.cc:1514] Processing DeleteTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:06 UTC) from {username='slave'} at 127.0.0.1:49252
I20260924 16:07:06.581576 18257 tablet_service.cc:1514] Processing DeleteTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:06 UTC) from {username='slave'} at 127.0.0.1:46398
I20260924 16:07:06.582149 18496 tablet_replica.cc:330] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a: stopping tablet replica
I20260924 16:07:06.582149 18495 tablet_replica.cc:330] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: stopping tablet replica
I20260924 16:07:06.582510 18389 tablet_service.cc:1514] Processing DeleteTablet for tablet 532e07d68e2a42a7aa19dfc101cfaad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:06 UTC) from {username='slave'} at 127.0.0.1:37304
I20260924 16:07:06.582571 18496 raft_consensus.cc:2229] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:06.582576 18495 raft_consensus.cc:2229] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:06.582934 18496 raft_consensus.cc:2258] T 532e07d68e2a42a7aa19dfc101cfaad0 P 7568378ef2a64afa8632ea4e88f27a0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:06.583130 18495 raft_consensus.cc:2258] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:06.583839 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18063
I20260924 16:07:06.584627 18497 tablet_replica.cc:330] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: stopping tablet replica
I20260924 16:07:06.585002 18497 raft_consensus.cc:2229] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:06.585068 18495 ts_tablet_manager.cc:1905] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:06.585405 18497 raft_consensus.cc:2258] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:06.587001 18497 ts_tablet_manager.cc:1905] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:06.588686 18495 ts_tablet_manager.cc:1918] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:06.588860 18495 log.cc:1196] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/532e07d68e2a42a7aa19dfc101cfaad0
I20260924 16:07:06.589296 18495 ts_tablet_manager.cc:1939] T 532e07d68e2a42a7aa19dfc101cfaad0 P 8e394b474ae84d0ca2ec53eaf0ca8335: Deleting consensus metadata
I20260924 16:07:06.590097 18497 ts_tablet_manager.cc:1918] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:06.590210 18497 log.cc:1196] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/532e07d68e2a42a7aa19dfc101cfaad0
I20260924 16:07:06.590605 18497 ts_tablet_manager.cc:1939] T 532e07d68e2a42a7aa19dfc101cfaad0 P dd41a8fb09c6413c8265b09a1144ae3e: Deleting consensus metadata
I20260924 16:07:06.591100 17988 catalog_manager.cc:4841] TS 8e394b474ae84d0ca2ec53eaf0ca8335 (127.13.8.66:42401): tablet 532e07d68e2a42a7aa19dfc101cfaad0 (table test-table [id=f63c04d7f71d46628558d7f007ebe5df]) successfully deleted
W20260924 16:07:06.591828 17989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:43355 (error 108)
I20260924 16:07:06.592082 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18195
I20260924 16:07:06.592363 17989 catalog_manager.cc:4841] TS dd41a8fb09c6413c8265b09a1144ae3e (127.13.8.67:35115): tablet 532e07d68e2a42a7aa19dfc101cfaad0 (table test-table [id=f63c04d7f71d46628558d7f007ebe5df]) successfully deleted
I20260924 16:07:06.593101 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.592637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43355 (local address 127.0.0.1:49268)
0924 16:07:06.592916 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.592995 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260924 16:07:06.593309 17989 catalog_manager.cc:4584] TS 7568378ef2a64afa8632ea4e88f27a0a (127.13.8.65:43355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 532e07d68e2a42a7aa19dfc101cfaad0: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43355: connect: Connection refused (error 111)
I20260924 16:07:06.598464 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18327
I20260924 16:07:06.604497 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 17972
2026-09-24T16:07:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1037 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 16:07:06.626196 13345 mini_oidc.cc:152] Starting JWKS server
I20260924 16:07:06.628852 13345 webserver.cc:460] Webserver started at https://127.0.0.1:38887/ using document root <none> and password file <none>
I20260924 16:07:06.629026 13345 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 16:07:06.629313 13345 webserver.cc:460] Webserver started at http://127.0.0.1:34577/ using document root <none> and password file <none>
2026-09-24T16:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:06Z Disabled control of system clock
I20260924 16:07:06.638131 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46465
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:46385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:46465
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38887/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/testchainca.cert with env {}
W20260924 16:07:06.749248 18504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:06.749564 18504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:06.749632 18504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:06.753669 18504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:06.753769 18504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:06.753799 18504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:06.753840 18504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:06.753875 18504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 16:07:06.753912 18504 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 16:07:06.753944 18504 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38887/jwks.json
I20260924 16:07:06.759316 18504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:46465
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38887/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:06.760448 18504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:06.761493 18504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:06.770081 18512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:06.770081 18504 server_base.cc:1034] running on GCE node
W20260924 16:07:06.770104 18509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:06.770145 18510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:06.770982 18504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:06.772043 18504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:06.773200 18504 hybrid_clock.cc:648] HybridClock initialized: now 1790266026773185 us; error 30 us; skew 500 ppm
I20260924 16:07:06.775125 18504 webserver.cc:460] Webserver started at http://127.13.8.126:37487/ using document root <none> and password file <none>
I20260924 16:07:06.775708 18504 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:06.775776 18504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:06.775990 18504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:06.778530 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "7c86551339ff48078f00b3657312f49e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "84122bed7b406f77612f82a70ed74f94"
server_key_iv: "ad2bced69c5c1e1078d08266ec3dbe50"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.779145 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "7c86551339ff48078f00b3657312f49e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "84122bed7b406f77612f82a70ed74f94"
server_key_iv: "ad2bced69c5c1e1078d08266ec3dbe50"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.782976 18504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:07:06.785342 18518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:06.786635 18504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:06.786779 18504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "7c86551339ff48078f00b3657312f49e"
format_stamp: "Formatted at 2026-09-24 16:07:06 on dist-test-slave-vj6n"
server_key: "84122bed7b406f77612f82a70ed74f94"
server_key_iv: "ad2bced69c5c1e1078d08266ec3dbe50"
server_key_version: "encryptionkey@0"
I20260924 16:07:06.786893 18504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:06.802274 18504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:06.802970 18504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:06.803166 18504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:06.810096 18504 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:46465
I20260924 16:07:06.810111 18570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:46465 every 8 connection(s)
I20260924 16:07:06.811080 18504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:06.813833 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18504
I20260924 16:07:06.813987 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:06.814298 13345 external_mini_cluster.cc:1428] Setting key ae3801c7516a455d4b05a88d24fd65be
I20260924 16:07:06.814612 18571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:06.820983 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e: Bootstrap starting.
I20260924 16:07:06.823355 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.815646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.13.8.126:46465)
0924 16:07:06.816307 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.816323 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.816357 (+ 34us) server_negotiation.cc:407] Connection header received
0924 16:07:06.816999 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.817016 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.817401 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.817732 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:06.818172 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.819184 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.820097 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.820565 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.820732 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:06.821634 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:06.821639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:06.821650 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:06.822017 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:06.822068 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:06.822088 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:06.822162 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:06.822353 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 16:07:06.822641 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":137,"threads_started":1}
I20260924 16:07:06.823583 18571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:06.824373 18571 log.cc:824] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:06.826176 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:07:06.826602 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e: No bootstrap required, opened a new log
I20260924 16:07:06.830477 18571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } }
I20260924 16:07:06.830693 18571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:06.830773 18571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c86551339ff48078f00b3657312f49e, State: Initialized, Role: FOLLOWER
I20260924 16:07:06.831215 18571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } }
I20260924 16:07:06.831398 18571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:06.831508 18571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:06.831620 18571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:06.832511 18571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } }
I20260924 16:07:06.832927 18571 leader_election.cc:302] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c86551339ff48078f00b3657312f49e; no voters:
I20260924 16:07:06.833300 18571 leader_election.cc:288] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:06.833415 18576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:06.833668 18576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [term 1 LEADER]: Becoming Leader. State: Replica: 7c86551339ff48078f00b3657312f49e, State: Running, Role: LEADER
I20260924 16:07:06.833962 18576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } }
I20260924 16:07:06.834743 18571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:06.835852 18577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c86551339ff48078f00b3657312f49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } } }
I20260924 16:07:06.835870 18578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c86551339ff48078f00b3657312f49e. Latest consensus state: current_term: 1 leader_uuid: "7c86551339ff48078f00b3657312f49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c86551339ff48078f00b3657312f49e" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46465 } } }
I20260924 16:07:06.836093 18577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:06.836122 18578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:06.837092 18585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:06.839917 18585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:06.845091 18585 catalog_manager.cc:1347] Generated new cluster ID: 356bb88883c94fc2b4528943cfe03fb0
I20260924 16:07:06.845155 18585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:06.862651 18585 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:06.863587 18585 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:06.871186 18585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c86551339ff48078f00b3657312f49e: Generated new TSK 0
I20260924 16:07:06.871833 18585 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 16:07:06.916879 18601 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 16:07:06.917007 18601 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 16:07:06.917522 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:06.891386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.13.8.126:46465)
0924 16:07:06.891534 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:06.891540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:06.891894 (+ 354us) server_negotiation.cc:407] Connection header received
0924 16:07:06.892124 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:06.892127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:06.892178 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:06.892264 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0924 16:07:06.892571 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.893098 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.894352 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.894567 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.894641 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:06.895030 (+ 389us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 16:07:06.917052 (+ 22022us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 16:07:06.917407 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260924 16:07:06.917630 18574 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 16:07:06.919083 18600 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:06.891415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:46465 (local address 127.0.0.1:34074)
0924 16:07:06.891796 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:06.891812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:06.891958 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:06.892306 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:06.892309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:06.892319 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0924 16:07:06.892456 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:06.892462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.893201 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.893205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:06.894203 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:06.894239 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:06.894673 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:06.894677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:06.894770 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:06.894828 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 16:07:06.917387 (+ 22559us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 16:07:06.917653 (+ 266us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.8.126:46465: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":159,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.8.126:46465: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 16:07:06.919853 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18504
2026-09-24T16:07:06Z chronyd exiting
I20260924 16:07:06.957576 13345 test_util.cc:175] -----------------------------------------------
I20260924 16:07:06.957670 13345 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790266001279336-13345-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 16:07:06.958846 13345 mini_oidc.cc:152] Starting JWKS server
I20260924 16:07:06.959708 13345 webserver.cc:460] Webserver started at https://127.0.0.1:41151/ using document root <none> and password file <none>
I20260924 16:07:06.959821 13345 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 16:07:06.960096 13345 webserver.cc:460] Webserver started at http://127.0.0.1:39313/ using document root <none> and password file <none>
2026-09-24T16:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:06Z Disabled control of system clock
I20260924 16:07:06.970119 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46117
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:46117
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41151/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/testchainca.cert with env {}
W20260924 16:07:07.081405 18607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:07.081722 18607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:07.081765 18607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:07.085845 18607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:07.085899 18607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:07.085914 18607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:07.085932 18607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:07.085947 18607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 16:07:07.085963 18607 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 16:07:07.085978 18607 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41151/jwks.json
I20260924 16:07:07.091360 18607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:46117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:46117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41151/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:07.092397 18607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:07.093375 18607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:07.101899 18615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:07.101923 18612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:07.101994 18607 server_base.cc:1034] running on GCE node
W20260924 16:07:07.101899 18613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:07.102607 18607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:07.103706 18607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:07.104899 18607 hybrid_clock.cc:648] HybridClock initialized: now 1790266027104887 us; error 23 us; skew 500 ppm
I20260924 16:07:07.106855 18607 webserver.cc:460] Webserver started at http://127.13.8.126:37109/ using document root <none> and password file <none>
I20260924 16:07:07.107424 18607 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:07.107514 18607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:07.107751 18607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:07.109664 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "3d20d99544e8438082d2a4bfc19fb4bb"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "1b2d6bf542c236184066847fe1e71a76"
server_key_iv: "9ea1f019ffb5f3a8bd757ff99c76516b"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.110394 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "3d20d99544e8438082d2a4bfc19fb4bb"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "1b2d6bf542c236184066847fe1e71a76"
server_key_iv: "9ea1f019ffb5f3a8bd757ff99c76516b"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.113867 18607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:07.116343 18621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:07.117549 18607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:07:07.117687 18607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "3d20d99544e8438082d2a4bfc19fb4bb"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "1b2d6bf542c236184066847fe1e71a76"
server_key_iv: "9ea1f019ffb5f3a8bd757ff99c76516b"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.117808 18607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:07.130865 18607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:07.131516 18607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:07.131721 18607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:07.138496 18607 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:46117
I20260924 16:07:07.138510 18673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:46117 every 8 connection(s)
I20260924 16:07:07.139461 18607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:07.142822 18674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:07.145644 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18607
I20260924 16:07:07.145768 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:07.146174 13345 external_mini_cluster.cc:1428] Setting key 310741df68e81c326a4cae55cbcd305c
I20260924 16:07:07.149421 18674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb: Bootstrap starting.
I20260924 16:07:07.152120 18674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:07.152935 18674 log.cc:824] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:07.155377 18674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb: No bootstrap required, opened a new log
I20260924 16:07:07.156235 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.147735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.13.8.126:46117)
0924 16:07:07.148403 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.148420 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.148456 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:07:07.149076 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.149103 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.149505 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.149872 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:07.150296 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.151249 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.152477 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.152956 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.153136 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.154041 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:07.154047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:07.154064 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:07.154606 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:07.154680 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:07.154708 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:07.154807 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:07.154955 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 16:07:07.155243 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":142,"threads_started":1}
I20260924 16:07:07.159327 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:07:07.159756 18674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } }
I20260924 16:07:07.160044 18674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:07.160137 18674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d20d99544e8438082d2a4bfc19fb4bb, State: Initialized, Role: FOLLOWER
I20260924 16:07:07.160642 18674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } }
I20260924 16:07:07.160805 18674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:07.160892 18674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:07.161015 18674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:07.161990 18674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } }
I20260924 16:07:07.162415 18674 leader_election.cc:302] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d20d99544e8438082d2a4bfc19fb4bb; no voters:
I20260924 16:07:07.162768 18674 leader_election.cc:288] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:07.162884 18679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:07.163123 18679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [term 1 LEADER]: Becoming Leader. State: Replica: 3d20d99544e8438082d2a4bfc19fb4bb, State: Running, Role: LEADER
I20260924 16:07:07.163457 18679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } }
I20260924 16:07:07.164073 18674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:07.165302 18680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } } }
I20260924 16:07:07.165443 18680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:07.165719 18681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d20d99544e8438082d2a4bfc19fb4bb. Latest consensus state: current_term: 1 leader_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20d99544e8438082d2a4bfc19fb4bb" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 46117 } } }
I20260924 16:07:07.165833 18681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:07.165915 18688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:07.169193 18688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:07.175251 18688 catalog_manager.cc:1347] Generated new cluster ID: e5457c6ad8cf4c23bcc914e5f3df99d3
I20260924 16:07:07.175348 18688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:07.187687 18688 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:07.188648 18688 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:07.196980 18688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d20d99544e8438082d2a4bfc19fb4bb: Generated new TSK 0
I20260924 16:07:07.197728 18688 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 16:07:07.223939 18704 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 16:07:07.224030 18704 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 16:07:07.224681 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.208914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.13.8.126:46117)
0924 16:07:07.209065 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.209072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.209227 (+ 155us) server_negotiation.cc:407] Connection header received
0924 16:07:07.209413 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.209416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.209461 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.209544 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0924 16:07:07.209808 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.210393 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.211599 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.211817 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.211888 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.212208 (+ 320us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 16:07:07.224341 (+ 12133us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 16:07:07.224559 (+ 218us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260924 16:07:07.224772 18677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 16:07:07.224937 18703 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:07.208832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:46117 (local address 127.0.0.1:55066)
0924 16:07:07.209137 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:07.209152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:07.209263 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:07.209574 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:07.209578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:07.209586 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0924 16:07:07.209702 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.209708 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.210494 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.210498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.211482 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.211493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.211923 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.211927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.212016 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.212043 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 16:07:07.224553 (+ 12510us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 16:07:07.224757 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.8.126:46117: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
I20260924 16:07:07.225672 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18607
2026-09-24T16:07:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 16:07:07.265720 13345 mini_oidc.cc:152] Starting JWKS server
I20260924 16:07:07.266656 13345 webserver.cc:460] Webserver started at https://127.0.0.1:38793/ using document root <none> and password file <none>
I20260924 16:07:07.266770 13345 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 16:07:07.267056 13345 webserver.cc:460] Webserver started at http://127.0.0.1:37643/ using document root <none> and password file <none>
2026-09-24T16:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:07Z Disabled control of system clock
I20260924 16:07:07.275827 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40397
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:40397
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38793/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 16:07:07.384775 18710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:07.385053 18710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:07.385108 18710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:07.389074 18710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:07.389127 18710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:07.389142 18710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:07.389175 18710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:07.389190 18710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 16:07:07.389206 18710 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 16:07:07.389219 18710 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38793/jwks.json
I20260924 16:07:07.393953 18710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:40397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38793/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:07.395061 18710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:07.396395 18710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:07.404871 18716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:07.404922 18710 server_base.cc:1034] running on GCE node
W20260924 16:07:07.404867 18715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:07.404884 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:07.405728 18710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:07.406836 18710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:07.408023 18710 hybrid_clock.cc:648] HybridClock initialized: now 1790266027407992 us; error 41 us; skew 500 ppm
I20260924 16:07:07.409809 18710 webserver.cc:460] Webserver started at http://127.13.8.126:46185/ using document root <none> and password file <none>
I20260924 16:07:07.410434 18710 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:07.410518 18710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:07.410796 18710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:07.412757 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "dc9a241157314801bb400a18ce7607d4"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "da65139bb53ffe41c0b54bfb78581ab2"
server_key_iv: "89ac66c44206f56489d330434ffc72f7"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.413300 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "dc9a241157314801bb400a18ce7607d4"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "da65139bb53ffe41c0b54bfb78581ab2"
server_key_iv: "89ac66c44206f56489d330434ffc72f7"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.417119 18710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260924 16:07:07.419488 18724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:07.420698 18710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:07.420854 18710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "dc9a241157314801bb400a18ce7607d4"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "da65139bb53ffe41c0b54bfb78581ab2"
server_key_iv: "89ac66c44206f56489d330434ffc72f7"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.420989 18710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:07.432376 18710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:07.433050 18710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:07.433249 18710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:07.440037 18710 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:40397
I20260924 16:07:07.440043 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:40397 every 8 connection(s)
I20260924 16:07:07.440982 18710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:07.441329 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18710
I20260924 16:07:07.441461 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:07.441825 13345 external_mini_cluster.cc:1428] Setting key f04f39b19f15d46bea9f61d152723098
I20260924 16:07:07.444258 18777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:07.450812 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4: Bootstrap starting.
I20260924 16:07:07.451103 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.443098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.13.8.126:40397)
0924 16:07:07.443714 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.443729 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.443765 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:07:07.444411 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.444437 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.444828 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.445104 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:07.445457 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.446461 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.447690 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.448160 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.448285 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.449264 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:07.449268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:07.449280 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:07.449671 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:07.449729 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:07.449749 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:07.449824 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:07.450009 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 16:07:07.450338 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":141,"threads_started":1}
I20260924 16:07:07.453500 18777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:07.454144 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:07:07.454445 18777 log.cc:824] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:07.456616 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4: No bootstrap required, opened a new log
I20260924 16:07:07.459981 18777 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } }
I20260924 16:07:07.460242 18777 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:07.460291 18777 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9a241157314801bb400a18ce7607d4, State: Initialized, Role: FOLLOWER
I20260924 16:07:07.460693 18777 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } }
I20260924 16:07:07.460811 18777 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:07.460851 18777 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:07.460920 18777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:07.462085 18777 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } }
I20260924 16:07:07.462438 18777 leader_election.cc:302] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9a241157314801bb400a18ce7607d4; no voters:
I20260924 16:07:07.462692 18777 leader_election.cc:288] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:07.462882 18782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:07.463100 18782 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [term 1 LEADER]: Becoming Leader. State: Replica: dc9a241157314801bb400a18ce7607d4, State: Running, Role: LEADER
I20260924 16:07:07.463405 18782 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } }
I20260924 16:07:07.463917 18777 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:07.465034 18783 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9a241157314801bb400a18ce7607d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } } }
I20260924 16:07:07.465178 18783 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:07.465479 18784 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9a241157314801bb400a18ce7607d4. Latest consensus state: current_term: 1 leader_uuid: "dc9a241157314801bb400a18ce7607d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9a241157314801bb400a18ce7607d4" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40397 } } }
I20260924 16:07:07.465596 18784 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:07.465698 18791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:07.468724 18791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:07.474195 18791 catalog_manager.cc:1347] Generated new cluster ID: fc9d7dd2d52b41f89d45fab2570eb99a
I20260924 16:07:07.474283 18791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:07.495038 18791 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:07.495973 18791 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:07.499961 18791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9a241157314801bb400a18ce7607d4: Generated new TSK 0
I20260924 16:07:07.500741 18791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:07.520704 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.515649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.13.8.126:40397)
0924 16:07:07.515790 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.515794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.515966 (+ 172us) server_negotiation.cc:407] Connection header received
0924 16:07:07.516096 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.516099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.516153 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.516275 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0924 16:07:07.516480 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.517295 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.518161 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.518523 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.518625 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.518942 (+ 317us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 16:07:07.520349 (+ 1407us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 16:07:07.520571 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42428: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260924 16:07:07.520781 18780 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42428: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 16:07:07.522401 18806 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:07.515577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40397 (local address 127.0.0.1:42428)
0924 16:07:07.515880 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:07.515893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:07.515989 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:07.516279 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:07.516281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:07.516288 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0924 16:07:07.516383 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.516388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.517417 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.517420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.518047 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.518055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.518665 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.518668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.518737 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.518760 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 16:07:07.520518 (+ 1758us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 16:07:07.522233 (+ 1715us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.8.126:40397: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":1522,"thread_start_us":118,"threads_started":1}
I20260924 16:07:07.523105 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18710
E20260924 16:07:07.531755 18807 webserver.cc:533] Webserver: sslize failed:
E20260924 16:07:07.531818 18807 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T16:07:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T16:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:07Z Disabled control of system clock
I20260924 16:07:07.571652 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40567
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:40567
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/testchainca.cert with env {}
W20260924 16:07:07.683983 18811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:07.684255 18811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:07.684312 18811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:07.688289 18811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:07.688337 18811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:07.688352 18811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:07.688369 18811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:07.688382 18811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 16:07:07.688397 18811 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 16:07:07.688410 18811 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 16:07:07.693341 18811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:40567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:40567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:07.694392 18811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:07.695351 18811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:07.703945 18816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:07.703924 18819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:07.703969 18817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:07.704092 18811 server_base.cc:1034] running on GCE node
I20260924 16:07:07.704728 18811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:07.705756 18811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:07.706931 18811 hybrid_clock.cc:648] HybridClock initialized: now 1790266027706899 us; error 40 us; skew 500 ppm
I20260924 16:07:07.708813 18811 webserver.cc:460] Webserver started at http://127.13.8.126:45935/ using document root <none> and password file <none>
I20260924 16:07:07.709383 18811 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:07.709472 18811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:07.709725 18811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:07.711844 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "ecf9cde51d7e49a2b8ed9491439e38a6"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "7dfd813fd3135c29a36b9d57cac133d3"
server_key_iv: "5280eda055f9d7eb38f58bd119488357"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.712375 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "ecf9cde51d7e49a2b8ed9491439e38a6"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "7dfd813fd3135c29a36b9d57cac133d3"
server_key_iv: "5280eda055f9d7eb38f58bd119488357"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.715911 18811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:07:07.718295 18825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:07.719444 18811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:07.719583 18811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "ecf9cde51d7e49a2b8ed9491439e38a6"
format_stamp: "Formatted at 2026-09-24 16:07:07 on dist-test-slave-vj6n"
server_key: "7dfd813fd3135c29a36b9d57cac133d3"
server_key_iv: "5280eda055f9d7eb38f58bd119488357"
server_key_version: "encryptionkey@0"
I20260924 16:07:07.719689 18811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:07.745652 18811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:07.746484 18811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:07.746685 18811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:07.753571 18811 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:40567
I20260924 16:07:07.753568 18877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:40567 every 8 connection(s)
I20260924 16:07:07.754597 18811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:07.757208 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18811
I20260924 16:07:07.757339 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:07.757467 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:07.757673 13345 external_mini_cluster.cc:1428] Setting key 57d7ab15f93976038941b77de0eb19f9
I20260924 16:07:07.763912 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: Bootstrap starting.
I20260924 16:07:07.766330 18878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:07.767081 18878 log.cc:824] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:07.767309 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.759037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.13.8.126:40567)
0924 16:07:07.759710 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.759725 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.759759 (+ 34us) server_negotiation.cc:407] Connection header received
0924 16:07:07.760581 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.760598 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.760995 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.761351 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:07.761787 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.762827 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.763758 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.764278 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.764446 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.765390 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:07.765395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:07.765408 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:07.765833 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:07.765888 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:07.765913 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:07.766029 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:07.766192 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 16:07:07.766531 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
I20260924 16:07:07.769027 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: No bootstrap required, opened a new log
I20260924 16:07:07.770190 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:07:07.773010 18878 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } }
I20260924 16:07:07.773244 18878 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:07.773319 18878 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf9cde51d7e49a2b8ed9491439e38a6, State: Initialized, Role: FOLLOWER
I20260924 16:07:07.773798 18878 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } }
I20260924 16:07:07.773941 18878 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:07.774025 18878 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:07.774149 18878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:07.775233 18878 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } }
I20260924 16:07:07.775619 18878 leader_election.cc:302] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecf9cde51d7e49a2b8ed9491439e38a6; no voters:
I20260924 16:07:07.775938 18878 leader_election.cc:288] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:07.776099 18883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:07.776516 18883 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [term 1 LEADER]: Becoming Leader. State: Replica: ecf9cde51d7e49a2b8ed9491439e38a6, State: Running, Role: LEADER
I20260924 16:07:07.776820 18883 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } }
I20260924 16:07:07.777348 18878 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:07.778656 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecf9cde51d7e49a2b8ed9491439e38a6. Latest consensus state: current_term: 1 leader_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } } }
I20260924 16:07:07.778702 18884 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf9cde51d7e49a2b8ed9491439e38a6" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 40567 } } }
I20260924 16:07:07.778826 18884 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:07.778777 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6 [sys.catalog]: This master's current role is: LEADER
W20260924 16:07:07.782189 18898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:07:07.782326 18898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:07:07.782405 18893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:07.783083 18893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:07.788671 18893 catalog_manager.cc:1347] Generated new cluster ID: 45b1636d38044b91bcf8c39715eb48ec
I20260924 16:07:07.788754 18893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:07.811296 18893 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:07.812208 18893 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:07.816717 18893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecf9cde51d7e49a2b8ed9491439e38a6: Generated new TSK 0
I20260924 16:07:07.817519 18893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:07.898844 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:07.829875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.13.8.126:40567)
0924 16:07:07.830014 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:07.830018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:07.830212 (+ 194us) server_negotiation.cc:407] Connection header received
0924 16:07:07.830396 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:07.830399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:07.830444 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:07.830528 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0924 16:07:07.830782 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.831291 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.832500 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.832708 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.832787 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.833138 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 16:07:07.898289 (+ 65151us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 16:07:07.898678 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59876: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":53}
W20260924 16:07:07.898895 18907 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:07.829819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:40567 (local address 127.0.0.1:59876)
0924 16:07:07.830116 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:07.830131 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:07.830270 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:07.830550 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:07.830553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:07.830569 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0924 16:07:07.830680 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.830686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.831400 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.831404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.832389 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:07.832400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:07.832844 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:07.832848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:07.832950 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:07.832977 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 16:07:07.898549 (+ 65572us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 16:07:07.898733 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.8.126:40567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
W20260924 16:07:07.898950 18881 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59876: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260924 16:07:07.899590 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18811
2026-09-24T16:07:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (355 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (126 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (125 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:08 dist-test-slave-vj6n krb5kdc[18919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:08 dist-test-slave-vj6n krb5kdc[18919](info): set up 2 sockets
Sep 24 16:07:08 dist-test-slave-vj6n krb5kdc[18919](info): commencing operation
krb5kdc: starting...
W20260924 16:07:09.183445 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:09.307435 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41315
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:09.416435 18932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:09.416726 18932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:09.416805 18932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:09.420760 18932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:09.420848 18932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:09.420886 18932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:09.420943 18932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:09.420981 18932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:09.426703 18932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:09.427769 18932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:09.428797 18932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:09.437152 18937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.437161 18938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.437176 18940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:09.437310 18932 server_base.cc:1034] running on GCE node
I20260924 16:07:09.437894 18932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:09.439080 18932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:09.440246 18932 hybrid_clock.cc:648] HybridClock initialized: now 1790266029440219 us; error 39 us; skew 500 ppm
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:09.443444 18932 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:09.444314 18932 webserver.cc:460] Webserver started at http://127.13.8.126:40619/ using document root <none> and password file <none>
I20260924 16:07:09.444829 18932 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:09.444886 18932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:09.445045 18932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:09.447049 18932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "85c5e2cb08a442ceab75b5265c56e081"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "ec10aa2803fab68440d3f4922526f7b5"
server_key_iv: "c080bf8640f6c9f3a442075683e153a0"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.447615 18932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "85c5e2cb08a442ceab75b5265c56e081"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "ec10aa2803fab68440d3f4922526f7b5"
server_key_iv: "c080bf8640f6c9f3a442075683e153a0"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.451181 18932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:07:09.453532 18947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.454828 18932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:09.454922 18932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "85c5e2cb08a442ceab75b5265c56e081"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "ec10aa2803fab68440d3f4922526f7b5"
server_key_iv: "c080bf8640f6c9f3a442075683e153a0"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.454999 18932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:09.480823 18932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:09.483920 18932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:09.484067 18932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:09.490928 18932 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41315
I20260924 16:07:09.490947 18999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41315 every 8 connection(s)
I20260924 16:07:09.491837 18932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:09.493420 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 18932
I20260924 16:07:09.493510 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:09.493810 13345 external_mini_cluster.cc:1428] Setting key c63a800229d09cae6af9deb80f0cdd9f
I20260924 16:07:09.495043 19000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:09.502347 19000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081: Bootstrap starting.
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:09.504895 19000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:09.505805 19000 log.cc:824] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:09.508097 19000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081: No bootstrap required, opened a new log
I20260924 16:07:09.508884 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:09.495246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.13.8.126:41315)
0924 16:07:09.495654 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:09.495673 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:09.495714 (+ 41us) server_negotiation.cc:407] Connection header received
0924 16:07:09.496680 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:09.496707 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:09.497140 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:09.497509 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:09.498155 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.499662 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.500521 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.501239 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.501409 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:09.503915 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:09.503921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:09.503938 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:09.503969 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:09.505887 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:09.506363 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:09.506367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:09.506372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:09.506477 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:09.506759 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:09.506762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:09.506764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:09.507234 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:09.507450 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:09.507658 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 16:07:09.507946 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":111,"threads_started":1}
I20260924 16:07:09.512105 19000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } }
I20260924 16:07:09.512313 19000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:09.512347 19000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c5e2cb08a442ceab75b5265c56e081, State: Initialized, Role: FOLLOWER
I20260924 16:07:09.512750 19000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } }
I20260924 16:07:09.512871 19000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:09.512916 19000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:09.512984 19000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:09.513773 19000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } }
I20260924 16:07:09.514061 19000 leader_election.cc:302] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85c5e2cb08a442ceab75b5265c56e081; no voters:
I20260924 16:07:09.514380 19000 leader_election.cc:288] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:09.514509 19005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:09.514741 19005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [term 1 LEADER]: Becoming Leader. State: Replica: 85c5e2cb08a442ceab75b5265c56e081, State: Running, Role: LEADER
I20260924 16:07:09.515038 19005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } }
I20260924 16:07:09.515663 19000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:09.516664 19006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85c5e2cb08a442ceab75b5265c56e081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } } }
I20260924 16:07:09.516806 19006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:09.517200 19014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:09.517248 19007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85c5e2cb08a442ceab75b5265c56e081. Latest consensus state: current_term: 1 leader_uuid: "85c5e2cb08a442ceab75b5265c56e081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c5e2cb08a442ceab75b5265c56e081" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41315 } } }
I20260924 16:07:09.517349 19007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:09.520753 19014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:09.526530 19014 catalog_manager.cc:1347] Generated new cluster ID: 97f8e54f57e442ac82940b8739e69b86
I20260924 16:07:09.526647 19014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:09.537391 19014 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:09.538684 19014 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:09.549804 19014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85c5e2cb08a442ceab75b5265c56e081: Generated new TSK 0
I20260924 16:07:09.550784 19014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:09.580941 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:09.693953 19026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:09.694231 19026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:09.694332 19026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:09.698341 19026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:09.698419 19026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:09.698518 19026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:09.703667 19026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:09.704722 19026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:09.705796 19026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:09.715078 19032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.715088 19034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.715145 19031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:09.715400 19026 server_base.cc:1034] running on GCE node
I20260924 16:07:09.715794 19026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:09.716559 19026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:09.717717 19026 hybrid_clock.cc:648] HybridClock initialized: now 1790266029717700 us; error 25 us; skew 500 ppm
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:09.723939 19026 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:09.724756 19026 webserver.cc:460] Webserver started at http://127.13.8.65:43261/ using document root <none> and password file <none>
I20260924 16:07:09.725311 19026 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:09.725399 19026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:09.725629 19026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:09.727707 19026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "38ade8e546934b53a5a35e49e3de22e1"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "a0c28e535ef86d7865fd2669bcf69bfd"
server_key_iv: "6c457175b5fbc0f31b1ed166cd3af984"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.728261 19026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "38ade8e546934b53a5a35e49e3de22e1"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "a0c28e535ef86d7865fd2669bcf69bfd"
server_key_iv: "6c457175b5fbc0f31b1ed166cd3af984"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.731819 19026 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260924 16:07:09.734386 19041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.735649 19026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:09.735780 19026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "38ade8e546934b53a5a35e49e3de22e1"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "a0c28e535ef86d7865fd2669bcf69bfd"
server_key_iv: "6c457175b5fbc0f31b1ed166cd3af984"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.735886 19026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:09.767550 19026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:09.770666 19026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:09.770857 19026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:09.771486 19026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:09.772331 19026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:09.772399 19026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.772471 19026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:09.772521 19026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.781527 19026 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43357
I20260924 16:07:09.781550 19154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43357 every 8 connection(s)
I20260924 16:07:09.782488 19026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:09.788194 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19026
I20260924 16:07:09.788300 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:09.788571 13345 external_mini_cluster.cc:1428] Setting key 8ae8a47974d247524fd70c4396dcb1d7
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:09.796904 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:09.784340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:39501 (local address 127.13.8.126:41315)
0924 16:07:09.784520 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:09.784524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:09.785414 (+ 890us) server_negotiation.cc:407] Connection header received
0924 16:07:09.786806 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:09.786809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:09.786855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:09.786930 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:09.788158 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.788850 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.790012 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.790358 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.790477 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:09.793264 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:09.793268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:09.793270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:09.793292 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:09.794811 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:09.795322 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:09.795326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:09.795328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:09.795417 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:09.795773 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:09.795778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:09.795780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:09.795971 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:09.796080 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:09.796587 (+ 507us) server_negotiation.cc:299] Negotiation successful
0924 16:07:09.796730 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 16:07:09.797725 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:09.784811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41315 (local address 127.13.8.65:39501)
0924 16:07:09.785255 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:09.785299 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:09.786599 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:09.787094 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:09.787102 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:09.787452 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:09.787962 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:09.787984 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.788989 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.788992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:09.789883 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:09.789892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:09.790519 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:09.790522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:09.790684 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:09.791382 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:09.791400 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:09.793084 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:09.794945 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:09.794950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:09.794962 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:09.795208 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:09.795527 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:09.795530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:09.795531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:09.795670 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:09.796080 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:09.796088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:09.796351 (+ 263us) client_negotiation.cc:769] Sending connection context
0924 16:07:09.796546 (+ 195us) client_negotiation.cc:240] Negotiation successful
0924 16:07:09.796869 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:09.798980 19155 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41315
I20260924 16:07:09.799255 19155 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:09.799927 19155 heartbeater.cc:511] Master 127.13.8.126:41315 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:09.801645 18964 ts_manager.cc:194] Registered new tserver with Master: 38ade8e546934b53a5a35e49e3de22e1 (127.13.8.65:43357)
I20260924 16:07:09.803315 18964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:39501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:09.817905 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:09.927124 19160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:09.927388 19160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:09.927470 19160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:09.931509 19160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:09.931563 19160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:09.931668 19160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:09.936967 19160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:09.938001 19160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:09.939127 19160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:09.949545 19168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.949529 19165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:09.949524 19166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:09.949771 19160 server_base.cc:1034] running on GCE node
I20260924 16:07:09.950197 19160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:09.950909 19160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:09.952098 19160 hybrid_clock.cc:648] HybridClock initialized: now 1790266029952079 us; error 27 us; skew 500 ppm
Sep 24 16:07:09 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:09.954944 19160 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:09.955945 19160 webserver.cc:460] Webserver started at http://127.13.8.66:35089/ using document root <none> and password file <none>
I20260924 16:07:09.956599 19160 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:09.956698 19160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:09.956943 19160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:09.959136 19160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "d864efa7a36141d386b86d1bc4a895a2"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "d7a62c4648fc2d06b18472ee58cd7cf2"
server_key_iv: "246e307a8df004eaa4d5fa74cb452e17"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.959734 19160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "d864efa7a36141d386b86d1bc4a895a2"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "d7a62c4648fc2d06b18472ee58cd7cf2"
server_key_iv: "246e307a8df004eaa4d5fa74cb452e17"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.963631 19160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:07:09.966171 19175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.967499 19160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:09.967620 19160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "d864efa7a36141d386b86d1bc4a895a2"
format_stamp: "Formatted at 2026-09-24 16:07:09 on dist-test-slave-vj6n"
server_key: "d7a62c4648fc2d06b18472ee58cd7cf2"
server_key_iv: "246e307a8df004eaa4d5fa74cb452e17"
server_key_version: "encryptionkey@0"
I20260924 16:07:09.967713 19160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:09.984525 19160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:09.987975 19160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:09.988135 19160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:09.988713 19160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:09.989509 19160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:09.989570 19160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.989611 19160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:09.989634 19160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:09.998898 19160 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:37171
I20260924 16:07:09.998920 19288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:37171 every 8 connection(s)
I20260924 16:07:09.999981 19160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:10.003751 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19160
I20260924 16:07:10.003851 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:10.004119 13345 external_mini_cluster.cc:1428] Setting key fd8c066c62d6072c9bae58c472e756d8
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:10.014842 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.002330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:38189 (local address 127.13.8.126:41315)
0924 16:07:10.002474 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.002477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.003339 (+ 862us) server_negotiation.cc:407] Connection header received
0924 16:07:10.004902 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.004906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.004959 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.005051 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.006269 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.006797 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.007949 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.008199 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.008313 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.010864 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:10.010870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:10.010875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:10.010908 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:10.012608 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.013179 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.013186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.013189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.013282 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.013628 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.013634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.013638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.013895 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:10.014007 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:10.014576 (+ 569us) server_negotiation.cc:299] Negotiation successful
0924 16:07:10.014709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:10.015923 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.002691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41315 (local address 127.13.8.66:38189)
0924 16:07:10.003178 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:10.003222 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:10.004681 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:10.005227 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:10.005235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:10.005602 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:10.006093 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:10.006107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.006926 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.006929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:10.007821 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:10.007830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.008353 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.008357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:10.008533 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.009175 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:10.009192 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:10.010691 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:10.012741 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:10.012746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:10.012758 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:10.013064 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:10.013383 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:10.013385 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:10.013386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:10.013521 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:10.014022 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:10.014028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:10.014355 (+ 327us) client_negotiation.cc:769] Sending connection context
0924 16:07:10.014521 (+ 166us) client_negotiation.cc:240] Negotiation successful
0924 16:07:10.014899 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:07:10.017319 19289 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41315
I20260924 16:07:10.017581 19289 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:10.018208 19289 heartbeater.cc:511] Master 127.13.8.126:41315 requested a full tablet report, sending...
I20260924 16:07:10.019373 18964 ts_manager.cc:194] Registered new tserver with Master: d864efa7a36141d386b86d1bc4a895a2 (127.13.8.66:37171)
I20260924 16:07:10.020303 18964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:38189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:10.031550 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:10.143450 19294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:10.143702 19294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:10.143744 19294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:10.147742 19294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:10.147797 19294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:10.147855 19294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:10.152916 19294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:10.153863 19294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:10.154857 19294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:10.164359 19302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:10.164394 19299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:10.164498 19294 server_base.cc:1034] running on GCE node
W20260924 16:07:10.164443 19300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:10.164940 19294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:10.165580 19294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:10.166769 19294 hybrid_clock.cc:648] HybridClock initialized: now 1790266030166750 us; error 26 us; skew 500 ppm
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.169548 19294 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:10.170507 19294 webserver.cc:460] Webserver started at http://127.13.8.67:37253/ using document root <none> and password file <none>
I20260924 16:07:10.171017 19294 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:10.171083 19294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:10.171241 19294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:10.173275 19294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "e71e42c70d5f46c184cd68ad4e94e29c"
format_stamp: "Formatted at 2026-09-24 16:07:10 on dist-test-slave-vj6n"
server_key: "bf5fc06f04d959cf28e0396b57c9eb23"
server_key_iv: "9c37c7491bd484a63719d97f0f6704e3"
server_key_version: "encryptionkey@0"
I20260924 16:07:10.173806 19294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "e71e42c70d5f46c184cd68ad4e94e29c"
format_stamp: "Formatted at 2026-09-24 16:07:10 on dist-test-slave-vj6n"
server_key: "bf5fc06f04d959cf28e0396b57c9eb23"
server_key_iv: "9c37c7491bd484a63719d97f0f6704e3"
server_key_version: "encryptionkey@0"
I20260924 16:07:10.177417 19294 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:07:10.179808 19309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:10.180996 19294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:10.181121 19294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "e71e42c70d5f46c184cd68ad4e94e29c"
format_stamp: "Formatted at 2026-09-24 16:07:10 on dist-test-slave-vj6n"
server_key: "bf5fc06f04d959cf28e0396b57c9eb23"
server_key_iv: "9c37c7491bd484a63719d97f0f6704e3"
server_key_version: "encryptionkey@0"
I20260924 16:07:10.181217 19294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:10.213254 19294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:10.216770 19294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:10.216919 19294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:10.217561 19294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:10.218442 19294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:10.218490 19294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:10.218532 19294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:10.218557 19294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:10.227121 19294 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:32855
I20260924 16:07:10.227141 19422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:32855 every 8 connection(s)
I20260924 16:07:10.228029 19294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:10.238054 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19294
I20260924 16:07:10.238188 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:10.238523 13345 external_mini_cluster.cc:1428] Setting key 9575ea452ef373e502ca13417de3c109
I20260924 16:07:10.243059 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.229904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:41195 (local address 127.13.8.126:41315)
0924 16:07:10.230087 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.230092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.230953 (+ 861us) server_negotiation.cc:407] Connection header received
0924 16:07:10.232367 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.232370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.232421 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.232511 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.233631 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.234296 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.235498 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.235717 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.235814 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.238466 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:10.238469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:10.238471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:10.238496 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:10.240393 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.241055 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.241061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.241065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.241159 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.241643 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.241646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.241647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.241856 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:10.241941 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:10.242651 (+ 710us) server_negotiation.cc:299] Negotiation successful
0924 16:07:10.242828 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 16:07:10.244064 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.230319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41315 (local address 127.13.8.67:41195)
0924 16:07:10.230791 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:10.230842 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:10.232142 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:10.232668 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:10.232676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:10.233018 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:10.233496 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:10.233509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.234435 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.234442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:10.235351 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:10.235361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.235855 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.235858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:10.235988 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.236573 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:10.236590 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:10.238263 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:10.240513 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:10.240522 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:10.240539 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:10.240928 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:10.241304 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:10.241309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:10.241313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:10.241526 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:10.241968 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:10.241976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:10.242342 (+ 366us) client_negotiation.cc:769] Sending connection context
0924 16:07:10.242571 (+ 229us) client_negotiation.cc:240] Negotiation successful
0924 16:07:10.242934 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260924 16:07:10.245957 19423 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41315
I20260924 16:07:10.246207 19423 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:10.246809 19423 heartbeater.cc:511] Master 127.13.8.126:41315 requested a full tablet report, sending...
I20260924 16:07:10.248134 18964 ts_manager.cc:194] Registered new tserver with Master: e71e42c70d5f46c184cd68ad4e94e29c (127.13.8.67:32855)
I20260924 16:07:10.248708 18964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:41195
I20260924 16:07:10.252871 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/test-admin.keytab.
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.268625 13345 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 16:07:10.271737 19430 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:10.270112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.13.8.65:43357)
0924 16:07:10.270555 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.270571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.270596 (+ 25us) server_negotiation.cc:407] Connection header received
0924 16:07:10.270646 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.270651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.270842 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.270963 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.271495 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35820: BlockingRecv error: recv got EOF from 127.0.0.1:35820 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":162,"threads_started":1}
W20260924 16:07:10.272095 13345 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc)
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.273175 13345 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.274114 13345 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:07:10.283600 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.275046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.13.8.65:43357)
0924 16:07:10.275201 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.275204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.275264 (+ 60us) server_negotiation.cc:407] Connection header received
0924 16:07:10.275369 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.275371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.275408 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.275471 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.276032 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.276822 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.277649 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.278006 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.278085 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.280244 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:10.280252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:10.280285 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:10.280307 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:10.281875 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.282309 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.282314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.282315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.282391 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.282669 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.282671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.282672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.282910 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:10.283068 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:10.283246 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 16:07:10.283420 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260924 16:07:10.287469 19430 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:10.286613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.13.8.65:43357)
0924 16:07:10.286774 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.286777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.286807 (+ 30us) server_negotiation.cc:407] Connection header received
0924 16:07:10.286896 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.286898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.286933 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.286995 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.287373 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35832: BlockingRecv error: recv got EOF from 127.0.0.1:35832 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 16:07:10.287580 13345 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc' of type FILE
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.288589 13345 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.289419 13345 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:07:10.297626 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.290260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.13.8.65:43357)
0924 16:07:10.290375 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.290380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.290460 (+ 80us) server_negotiation.cc:407] Connection header received
0924 16:07:10.290623 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.290626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.290659 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.290741 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.291239 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.291766 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.292547 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.292777 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.292885 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.295145 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:10.295148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:10.295150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:10.295169 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:10.296336 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.296721 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.296723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.296725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.296793 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.297013 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.297015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.297016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.297156 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:10.297245 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:10.297437 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 16:07:10.297521 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260924 16:07:10.300030 19430 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:10.299218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35848 (local address 127.13.8.65:43357)
0924 16:07:10.299341 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.299351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.299364 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:10.299463 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.299465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.299495 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.299554 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.299923 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35848: BlockingRecv error: recv got EOF from 127.0.0.1:35848 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260924 16:07:10.300089 13345 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790266001279336-13345-0/krb5kdc/krb5cc' of type FILE
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.300967 13345 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:10.301812 13345 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[18919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.65@KRBTEST.COM
I20260924 16:07:10.309757 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:10.302695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.13.8.65:43357)
0924 16:07:10.302801 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:10.302804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:10.302917 (+ 113us) server_negotiation.cc:407] Connection header received
0924 16:07:10.303005 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:10.303006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:10.303035 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:10.303093 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:10.303632 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.304095 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.304822 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:10.305015 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:10.305087 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:10.307218 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:10.307221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:10.307222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:10.307242 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:10.308402 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.308786 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.308788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.308790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.308855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:10.309073 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:10.309075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:10.309076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:10.309212 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:10.309303 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:10.309564 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 16:07:10.309644 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:07:10.310827 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19026
I20260924 16:07:10.317435 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19160
I20260924 16:07:10.323828 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19294
I20260924 16:07:10.329427 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 18932
2026-09-24T16:07:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2179 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[19443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[19443](info): set up 2 sockets
Sep 24 16:07:10 dist-test-slave-vj6n krb5kdc[19443](info): commencing operation
krb5kdc: starting...
W20260924 16:07:11.403374 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:11 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab.
Entry for principal oryx/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab.
I20260924 16:07:11.521226 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:39979
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab
--principal=oryx/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:39979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:11.630978 19456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:11.631222 19456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:11.631263 19456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:11.635059 19456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:11.635102 19456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:11.635118 19456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:11.635134 19456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:11.635149 19456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:11.640424 19456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:39979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab
--principal=oryx/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:39979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:11.641417 19456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:11.642441 19456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:11.650733 19461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:11.650765 19462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:11.650765 19464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:11.650802 19456 server_base.cc:1034] running on GCE node
I20260924 16:07:11.651477 19456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:11.652570 19456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:11.653755 19456 hybrid_clock.cc:648] HybridClock initialized: now 1790266031653741 us; error 26 us; skew 500 ppm
Sep 24 16:07:11 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:11.656431 19456 init.cc:377] Logged in from keytab as oryx/127.13.8.126@KRBTEST.COM (short username oryx)
I20260924 16:07:11.657251 19456 webserver.cc:460] Webserver started at http://127.13.8.126:44175/ using document root <none> and password file <none>
I20260924 16:07:11.657805 19456 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:11.657892 19456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:11.658118 19456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:11.660120 19456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "bb4abb37975e499395cd083ddcfcd578"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "87e3d19d93c3718d60ecbf3fbe10f95b"
server_key_iv: "4b9c917f1bfeb80003f5bb794f4a5b8a"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.660674 19456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "bb4abb37975e499395cd083ddcfcd578"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "87e3d19d93c3718d60ecbf3fbe10f95b"
server_key_iv: "4b9c917f1bfeb80003f5bb794f4a5b8a"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.664266 19456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260924 16:07:11.666558 19471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:11.667727 19456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:11.667861 19456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "bb4abb37975e499395cd083ddcfcd578"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "87e3d19d93c3718d60ecbf3fbe10f95b"
server_key_iv: "4b9c917f1bfeb80003f5bb794f4a5b8a"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.667963 19456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:11.690181 19456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:11.694380 19456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:11.694590 19456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:11.703780 19456 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:39979
I20260924 16:07:11.704052 19523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:39979 every 8 connection(s)
I20260924 16:07:11.705078 19456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:11.707698 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19456
I20260924 16:07:11.707821 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:11.708130 13345 external_mini_cluster.cc:1428] Setting key adc9fbb7b9e95ba74ac69515943ad371
I20260924 16:07:11.708359 19524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:11.716986 19524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Bootstrap starting.
Sep 24 16:07:11 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:11.720278 19524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:11.721323 19524 log.cc:824] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:11.723919 19524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: No bootstrap required, opened a new log
I20260924 16:07:11.725113 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:11.709947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.13.8.126:39979)
0924 16:07:11.710374 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:11.710395 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:11.710437 (+ 42us) server_negotiation.cc:407] Connection header received
0924 16:07:11.711479 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:11.711506 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:11.712004 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:11.712380 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:11.713251 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:11.715095 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:11.716199 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:11.717092 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:11.717737 (+ 645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:11.720192 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:11.720198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:11.720223 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:11.720249 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:11.722120 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:11.722680 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:11.722686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:11.722692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:11.722811 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:11.723132 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:11.723136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:11.723138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:11.723545 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:11.723763 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:11.724032 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 16:07:11.724369 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":133,"threads_started":1}
I20260924 16:07:11.729805 19524 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } }
I20260924 16:07:11.730120 19524 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:11.730194 19524 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4abb37975e499395cd083ddcfcd578, State: Initialized, Role: FOLLOWER
I20260924 16:07:11.730804 19524 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } }
I20260924 16:07:11.730983 19524 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:11.731062 19524 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:11.731179 19524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:11.732563 19524 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } }
I20260924 16:07:11.733000 19524 leader_election.cc:302] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4abb37975e499395cd083ddcfcd578; no voters:
I20260924 16:07:11.733366 19524 leader_election.cc:288] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:11.733484 19529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:11.733758 19529 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 LEADER]: Becoming Leader. State: Replica: bb4abb37975e499395cd083ddcfcd578, State: Running, Role: LEADER
I20260924 16:07:11.734069 19529 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } }
I20260924 16:07:11.734704 19524 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:11.735795 19530 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } }
I20260924 16:07:11.735927 19530 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:11.736238 19531 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4abb37975e499395cd083ddcfcd578. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } }
I20260924 16:07:11.736320 19538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:11.736339 19531 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:11.739223 19538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:11.744858 19538 catalog_manager.cc:1347] Generated new cluster ID: ff812a99ca1b471c8013a8d455d27070
I20260924 16:07:11.744915 19538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:11.773409 19538 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:11.774750 19538 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:11.782730 19538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Generated new TSK 0
I20260924 16:07:11.783654 19538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
I20260924 16:07:11.821221 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:11.931900 19550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:11.932142 19550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:11.932181 19550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:11.936353 19550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:11.936403 19550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:11.936458 19550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:11.941612 19550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:11.942688 19550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:11.943706 19550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:11.952939 19558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:11.953065 19550 server_base.cc:1034] running on GCE node
W20260924 16:07:11.952939 19556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:11.952980 19555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:11.953598 19550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:11.954180 19550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:11.955375 19550 hybrid_clock.cc:648] HybridClock initialized: now 1790266031955364 us; error 21 us; skew 500 ppm
Sep 24 16:07:11 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:11.958070 19550 init.cc:377] Logged in from keytab as oryx/127.13.8.65@KRBTEST.COM (short username oryx)
I20260924 16:07:11.958927 19550 webserver.cc:460] Webserver started at http://127.13.8.65:36513/ using document root <none> and password file <none>
I20260924 16:07:11.959415 19550 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:11.959468 19550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:11.959620 19550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:11.961510 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "e85af61ce1e240c9aa23fe66d1f328aa"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "d6effcb6235000f91e409360f3bb4f7e"
server_key_iv: "67bf1c6ec83b82975d623dd985fd4a23"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.961990 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "e85af61ce1e240c9aa23fe66d1f328aa"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "d6effcb6235000f91e409360f3bb4f7e"
server_key_iv: "67bf1c6ec83b82975d623dd985fd4a23"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.965571 19550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:07:11.967878 19565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:11.969024 19550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260924 16:07:11.969159 19550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "e85af61ce1e240c9aa23fe66d1f328aa"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "d6effcb6235000f91e409360f3bb4f7e"
server_key_iv: "67bf1c6ec83b82975d623dd985fd4a23"
server_key_version: "encryptionkey@0"
I20260924 16:07:11.969261 19550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:11.991961 19550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:11.995355 19550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:11.995556 19550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:11.996170 19550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:11.997022 19550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:11.997105 19550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:11.997200 19550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:11.997251 19550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.006666 19550 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:41097
I20260924 16:07:12.006691 19678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:41097 every 8 connection(s)
I20260924 16:07:12.007686 19550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:12.017160 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19550
I20260924 16:07:12.017293 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:12.017654 13345 external_mini_cluster.cc:1428] Setting key fcc5d69c097a2ad3346ab94ad9916554
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:12.022579 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.009664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:44035 (local address 127.13.8.126:39979)
0924 16:07:12.009810 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.009814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.010739 (+ 925us) server_negotiation.cc:407] Connection header received
0924 16:07:12.012042 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.012045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.012100 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.012175 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.013356 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.013958 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.015213 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.015457 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.015558 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.018635 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.018638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.018640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.018663 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.020464 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.020991 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.020995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.020996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.021077 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.021432 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.021435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.021437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.021633 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.021742 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.022293 (+ 551us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.022419 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 16:07:12.023520 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.010042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:44035)
0924 16:07:12.010577 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.010622 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.011842 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.012339 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.012345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.012697 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.013188 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.013201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.014115 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.014117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.015097 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.015106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.015600 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.015603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.015749 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.016480 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.016510 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.018478 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.020584 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.020591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.020607 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.020862 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.021173 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.021177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.021179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.021329 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.021742 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.021750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.022009 (+ 259us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.022203 (+ 194us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.022603 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":182,"threads_started":1}
I20260924 16:07:12.024863 19679 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:12.025161 19679 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:12.025782 19679 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:12.027683 19488 ts_manager.cc:194] Registered new tserver with Master: e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097)
I20260924 16:07:12.028963 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:44035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
I20260924 16:07:12.045034 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:12.157224 19684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:12.157480 19684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:12.157526 19684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:12.161865 19684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:12.161918 19684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:12.162019 19684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:12.167279 19684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:12.168329 19684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:12.169344 19684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:12.179565 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.179656 19684 server_base.cc:1034] running on GCE node
W20260924 16:07:12.179672 19692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.179565 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.180265 19684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:12.180934 19684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:12.182125 19684 hybrid_clock.cc:648] HybridClock initialized: now 1790266032182111 us; error 25 us; skew 500 ppm
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:12.185185 19684 init.cc:377] Logged in from keytab as oryx/127.13.8.66@KRBTEST.COM (short username oryx)
I20260924 16:07:12.186431 19684 webserver.cc:460] Webserver started at http://127.13.8.66:33623/ using document root <none> and password file <none>
I20260924 16:07:12.187285 19684 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.187386 19684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.187660 19684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:12.190670 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "254277422fba4b1e99b1fd7c7d20b47b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "ed5bd1b63eb8cdfe398f75815d0e55fa"
server_key_iv: "d1dfac75daba913320798ddc6f3ec98e"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.191403 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "254277422fba4b1e99b1fd7c7d20b47b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "ed5bd1b63eb8cdfe398f75815d0e55fa"
server_key_iv: "d1dfac75daba913320798ddc6f3ec98e"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.195241 19684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:07:12.197671 19699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.198906 19684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:12.199035 19684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "254277422fba4b1e99b1fd7c7d20b47b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "ed5bd1b63eb8cdfe398f75815d0e55fa"
server_key_iv: "d1dfac75daba913320798ddc6f3ec98e"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.199142 19684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.222039 19684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:12.225163 19684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:12.225368 19684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:12.225991 19684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:12.226898 19684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:12.226945 19684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.227010 19684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:12.227042 19684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.235702 19684 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39245
I20260924 16:07:12.235726 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39245 every 8 connection(s)
I20260924 16:07:12.236671 19684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:12.241231 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19684
I20260924 16:07:12.241343 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:12.241695 13345 external_mini_cluster.cc:1428] Setting key c771fb9c1492e7d413a55fab77247fd0
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:12.254379 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.238538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:35423 (local address 127.13.8.126:39979)
0924 16:07:12.238672 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.238677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.239695 (+ 1018us) server_negotiation.cc:407] Connection header received
0924 16:07:12.241192 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.241198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.241289 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.241387 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.243345 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.244125 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.246078 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.246518 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.246627 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.250393 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.250397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.250400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.250449 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.252090 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.252701 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.252704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.252705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.252786 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.253178 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.253180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.253182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.253351 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.253423 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.254072 (+ 649us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.254172 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:12.255508 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.238898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:35423)
0924 16:07:12.239402 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.239474 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.240828 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.241741 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.241751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.242339 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.243106 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.243130 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.244260 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.244263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.245923 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.245935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.246816 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.246820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.247027 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.247872 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.247897 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.250170 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.252215 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.252224 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.252243 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.252601 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.252877 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.252881 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.252884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.253083 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.253459 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.253466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.253807 (+ 341us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.254030 (+ 223us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.254452 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260924 16:07:12.256680 19813 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:12.256938 19813 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:12.257442 19813 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.67@KRBTEST.COM" created.
I20260924 16:07:12.258612 19488 ts_manager.cc:194] Registered new tserver with Master: 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:12.259179 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:35423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
I20260924 16:07:12.271238 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:12.381485 19818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:12.381743 19818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:12.381789 19818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:12.385818 19818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:12.385874 19818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:12.385932 19818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:12.391547 19818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:12.392570 19818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:12.393536 19818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:12.403903 19823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.404000 19826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.404160 19824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.410751 19818 server_base.cc:1034] running on GCE node
I20260924 16:07:12.411235 19818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:12.411922 19818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:12.413102 19818 hybrid_clock.cc:648] HybridClock initialized: now 1790266032413066 us; error 49 us; skew 500 ppm
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:12.415766 19818 init.cc:377] Logged in from keytab as oryx/127.13.8.67@KRBTEST.COM (short username oryx)
I20260924 16:07:12.416724 19818 webserver.cc:460] Webserver started at http://127.13.8.67:44547/ using document root <none> and password file <none>
I20260924 16:07:12.417309 19818 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.417398 19818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.417634 19818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:12.419715 19818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "ed31894d5d5a4e8baf091ecbb6729e38"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "129f32e71104099ed6a5c74e4a3609d2"
server_key_iv: "8cfa1d8cebc652059688487914ee515a"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.420284 19818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "ed31894d5d5a4e8baf091ecbb6729e38"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "129f32e71104099ed6a5c74e4a3609d2"
server_key_iv: "8cfa1d8cebc652059688487914ee515a"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.423996 19818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:07:12.426390 19833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.427639 19818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:12.427780 19818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "ed31894d5d5a4e8baf091ecbb6729e38"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "129f32e71104099ed6a5c74e4a3609d2"
server_key_iv: "8cfa1d8cebc652059688487914ee515a"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.427887 19818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.446605 19818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:12.449738 19818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:12.449942 19818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:12.450626 19818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:12.451519 19818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:12.451567 19818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.451640 19818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:12.451673 19818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.460457 19818 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:45627
I20260924 16:07:12.460511 19946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:45627 every 8 connection(s)
I20260924 16:07:12.461387 19818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:12.467312 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19818
I20260924 16:07:12.467438 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:12.467758 13345 external_mini_cluster.cc:1428] Setting key 38b518cd3b2e23b4fc8fed64601c23f8
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:12.475546 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.463291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:56317 (local address 127.13.8.126:39979)
0924 16:07:12.463427 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.463431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.464265 (+ 834us) server_negotiation.cc:407] Connection header received
0924 16:07:12.465655 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.465658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.465723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.465821 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.467018 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.467697 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.469279 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.469491 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.469573 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.472152 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.472155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.472157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.472180 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.473637 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.474074 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.474077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.474078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.474149 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.474566 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.474570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.474572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.474772 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.474862 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.475326 (+ 464us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.475424 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:07:12.476346 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.463646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:56317)
0924 16:07:12.464119 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.464160 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.465432 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.465999 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.466007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.466410 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.466878 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.466890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.467819 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.467822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.469168 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.469178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.469643 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.469647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.469873 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.470491 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.470507 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.471991 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.473735 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.473739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.473751 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.473989 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.474264 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.474270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.474272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.474448 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.474872 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.474877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.475112 (+ 235us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.475260 (+ 148us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.475507 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260924 16:07:12.477500 19947 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:12.477741 19947 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:12.478180 19947 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
I20260924 16:07:12.479207 19488 ts_manager.cc:194] Registered new tserver with Master: ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:12.479765 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:56317
I20260924 16:07:12.481518 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.125.keytab.
Entry for principal oryx/127.13.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.125.keytab.
I20260924 16:07:12.507150 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:40327
--webserver_interface=127.13.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.125.keytab
--principal=oryx/127.13.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:39979,127.13.8.125:40327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:12.618706 19952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:12.618983 19952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:12.619052 19952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:12.623421 19952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:12.623497 19952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:12.623535 19952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:12.623580 19952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:12.623617 19952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:12.628901 19952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:39979,127.13.8.125:40327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.125.keytab
--principal=oryx/127.13.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:40327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
W20260924 16:07:12.629154 19952 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.8.126:39979,127.13.8.125:40327'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 16:07:12.630036 19952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:12.631170 19952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:12.639822 19960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.639904 19952 server_base.cc:1034] running on GCE node
W20260924 16:07:12.639839 19957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.639827 19958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.640556 19952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:12.641616 19952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:12.642768 19952 hybrid_clock.cc:648] HybridClock initialized: now 1790266032642754 us; error 25 us; skew 500 ppm
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:12.645505 19952 init.cc:377] Logged in from keytab as oryx/127.13.8.125@KRBTEST.COM (short username oryx)
I20260924 16:07:12.646437 19952 webserver.cc:460] Webserver started at http://127.13.8.125:37277/ using document root <none> and password file <none>
I20260924 16:07:12.647025 19952 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.647123 19952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.647358 19952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:12.649519 19952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/instance:
uuid: "6e60b5d591c7447fb748dd55655c0a12"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "2ba4958873040fe35f77b7d8fd79d394"
server_key_iv: "c7799212e8817b53132d37e1ef99ee88"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.650081 19952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal/instance:
uuid: "6e60b5d591c7447fb748dd55655c0a12"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "2ba4958873040fe35f77b7d8fd79d394"
server_key_iv: "c7799212e8817b53132d37e1ef99ee88"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.653785 19952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:12.656330 19967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.657599 19952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:12.657743 19952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "6e60b5d591c7447fb748dd55655c0a12"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "2ba4958873040fe35f77b7d8fd79d394"
server_key_iv: "c7799212e8817b53132d37e1ef99ee88"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.657858 19952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.668474 19952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:12.671832 19952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:12.672014 19952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:12.678807 19952 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.125:40327
I20260924 16:07:12.679068 20020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.125:40327 every 8 connection(s)
I20260924 16:07:12.679811 19952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
I20260924 16:07:12.682616 20021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:12.682741 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 19952
I20260924 16:07:12.687810 20021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.125@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:12.698763 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.684158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.13.8.125:40327)
0924 16:07:12.684553 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.684575 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.684614 (+ 39us) server_negotiation.cc:407] Connection header received
0924 16:07:12.685463 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.685488 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.685906 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.686279 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.687039 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.688503 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.689461 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.690155 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.690353 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.693411 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.693416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.693425 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.693449 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.695435 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.695889 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.695899 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.695903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.696006 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.696319 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.696325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.696328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.696890 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.697120 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.697439 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.697740 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260924 16:07:12.699437 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.689660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51416)
0924 16:07:12.690104 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.690155 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.690343 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.690690 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.690695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.691157 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.691293 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.691299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.692435 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.692439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.693256 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.693265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.693737 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.693753 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.693872 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.694360 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.694376 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.695968 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.697816 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.697824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.697830 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.698044 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.698350 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.698352 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.698353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.698422 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.698914 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.698922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.699040 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.699177 (+ 137us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.699315 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
I20260924 16:07:12.699484 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.689721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.13.8.126:39979)
0924 16:07:12.689865 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.689869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.690319 (+ 450us) server_negotiation.cc:407] Connection header received
0924 16:07:12.690443 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.690446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.690499 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.690661 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.691422 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.692295 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.693394 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.693621 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.693734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.696113 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.696118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.696121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.696147 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.697703 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.698142 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.698146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.698147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.698263 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.698510 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.698513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.698515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.698782 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.698899 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.699238 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.699349 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:07:12.700884 20021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.125@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
I20260924 16:07:12.708927 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.701331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36940)
0924 16:07:12.701444 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.701457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.701528 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.701688 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.701690 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.701806 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.701893 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.701897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.703105 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.703108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.703757 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.703768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.704169 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.704173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.704268 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.704646 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.704660 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.705878 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.707533 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.707536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.707538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.707741 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.708009 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.708012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.708014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.708091 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.708443 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.708447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.708531 (+ 84us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.708635 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.708765 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 16:07:12.708940 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.701416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.13.8.125:40327)
0924 16:07:12.701518 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.701521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.701532 (+ 11us) server_negotiation.cc:407] Connection header received
0924 16:07:12.701589 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.701591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.701622 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.701675 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.702005 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.702974 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.703861 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.704061 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.704135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.706000 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.706005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.706012 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.706031 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.707411 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.707835 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.707840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.707842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.707914 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.708174 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.708179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.708181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.708339 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.708442 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.708677 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.708818 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":39}
I20260924 16:07:12.712196 20021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Bootstrap starting.
I20260924 16:07:12.714955 20021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:12.715822 20021 log.cc:824] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:12.717808 20021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: No bootstrap required, opened a new log
I20260924 16:07:12.720775 20021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } }
I20260924 16:07:12.720984 20021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:12.721019 20021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e60b5d591c7447fb748dd55655c0a12, State: Initialized, Role: FOLLOWER
I20260924 16:07:12.721390 20021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } }
I20260924 16:07:12.721794 20027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } } }
I20260924 16:07:12.721925 20027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:12.722589 20021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 16:07:12.727188 20038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:07:12.727267 20038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:07:12.729166 19952 master_runner.cc:428] Detected that this master 6e60b5d591c7447fb748dd55655c0a12 is joining an existing cluster
I20260924 16:07:12.729226 19952 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.8.125:40327
I20260924 16:07:12.739639 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.732173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.13.8.125:40327)
0924 16:07:12.732293 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.732297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.732451 (+ 154us) server_negotiation.cc:407] Connection header received
0924 16:07:12.732546 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.732552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.732591 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.732692 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.733108 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.733570 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.734544 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.734744 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.734846 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.736288 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.736291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.736294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.736316 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.737941 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738460 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.738466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.738468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.738561 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738871 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.738874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.738876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.739038 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.739140 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.739368 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.739508 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":53}
I20260924 16:07:12.739653 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.731860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36950)
0924 16:07:12.732362 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.732373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.732479 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.732727 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.732730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.732910 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.733000 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.733005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.733676 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.733680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.734427 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.734437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.734870 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.734874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.734956 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.735626 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.735648 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.736083 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.738042 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.738051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.738311 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.738666 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.738672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.738769 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.739129 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.739132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.739210 (+ 78us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.739324 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.739457 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":71,"thread_start_us":94,"threads_started":1}
I20260924 16:07:12.740111 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.731545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.13.8.126:39979)
0924 16:07:12.731655 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.731660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.731833 (+ 173us) server_negotiation.cc:407] Connection header received
0924 16:07:12.731930 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.731933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.731982 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.732071 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.732669 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.733486 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.734803 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.735191 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.735270 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.736591 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.736595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.736598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.736624 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.738271 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738736 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.738739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.738741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.738827 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.739152 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.739155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.739157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.739367 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.739439 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.739814 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.739941 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 16:07:12.740586 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.731467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51418)
0924 16:07:12.731728 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.731747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.731848 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.732200 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.732204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.732412 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.732533 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.732540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.733629 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.733633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.734665 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.734677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.735433 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.735440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.735534 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.735996 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.736012 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.736431 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.738384 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.738392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.738592 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.738937 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.738940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.738942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.739031 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.739516 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.739519 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.739623 (+ 104us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.740287 (+ 664us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.740422 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":49,"thread_start_us":86,"threads_started":1}
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.126@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
I20260924 16:07:12.753185 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.744712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.13.8.125:40327)
0924 16:07:12.744823 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.744827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.745103 (+ 276us) server_negotiation.cc:407] Connection header received
0924 16:07:12.745305 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.745308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.745352 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.745440 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.746019 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.746542 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.747416 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.747631 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.747745 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.749818 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.749822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.749824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.749849 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.751433 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.751986 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.751992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.751995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.752081 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.752352 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.752356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.752358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.752533 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.752638 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.752950 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.753054 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:12.753216 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.744673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36956)
0924 16:07:12.744995 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.745032 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.745175 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.745475 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.745480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.745807 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.745907 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.745913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.746643 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.746647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.747297 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.747305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.747741 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.747744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.747855 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.748336 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.748348 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.749660 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.751540 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.751549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.751556 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.751839 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.752172 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.752176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.752178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.752263 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.752639 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.752648 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.752763 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.752901 (+ 138us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.753066 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260924 16:07:12.754918 19488 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.8.125:40327: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12"
member_type: NON_VOTER
last_known_addr {
host: "127.13.8.125"
port: 40327
}
attrs {
promote: true
}
}
dest_uuid: "bb4abb37975e499395cd083ddcfcd578"
cas_config_opid_index: -1
I20260924 16:07:12.755626 19488 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } }
I20260924 16:07:12.757817 20049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6e60b5d591c7447fb748dd55655c0a12 (127.13.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } }
I20260924 16:07:12.758447 19994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:12.759837 19994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 16:07:12.759889 20049 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.8.125:40327
I20260924 16:07:12.760401 20049 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } }
I20260924 16:07:12.760509 20049 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:12.760515 19952 master.cc:537] Master@127.13.8.125:40327 shutting down...
WARNING: no policy specified for oryx/127.13.8.124@KRBTEST.COM; defaulting to no policy
I20260924 16:07:12.762584 20048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.124@KRBTEST.COM" created.
W20260924 16:07:12.765511 19473 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260924 16:07:12.766355 19952 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 16:07:12.766453 19473 consensus_peers.cc:487] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 -> Peer 6e60b5d591c7447fb748dd55655c0a12 (127.13.8.125:40327): Couldn't send request to peer 6e60b5d591c7447fb748dd55655c0a12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 16:07:12.766755 19952 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:12.766878 19952 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: stopping tablet replica
I20260924 16:07:12.772372 19952 master.cc:552] Master@127.13.8.125:40327 shutdown complete.
I20260924 16:07:12.772420 19952 master_runner.cc:305] Clearing existing system tablet
I20260924 16:07:12.772681 19952 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.772737 19952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.774058 19952 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:12.775238 20054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.775475 19952 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.775596 19952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:12.775630 19952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "6e60b5d591c7447fb748dd55655c0a12"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "2ba4958873040fe35f77b7d8fd79d394"
server_key_iv: "c7799212e8817b53132d37e1ef99ee88"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.778393 19952 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.124.keytab.
Entry for principal oryx/127.13.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.124.keytab.
I20260924 16:07:12.781185 19952 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 16:07:12.781279 19952 log.cc:1196] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 16:07:12.781497 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:38309
--webserver_interface=127.13.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.124.keytab
--principal=oryx/127.13.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:07:12.781523 19952 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Deleting consensus metadata
I20260924 16:07:12.781694 19952 master_runner.cc:315] Copying system tablet from 127.13.8.126:39979
I20260924 16:07:12.782964 19952 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: tablet copy: Beginning tablet copy session from remote peer at address 127.13.8.126:39979
I20260924 16:07:12.791471 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.783679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.13.8.126:39979)
0924 16:07:12.783835 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:12.783840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:12.784112 (+ 272us) server_negotiation.cc:407] Connection header received
0924 16:07:12.784251 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:12.784254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:12.784311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:12.784403 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:12.784778 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.785610 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.786483 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.786717 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.786826 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.788135 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:12.788139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:12.788141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:12.788166 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:12.789805 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.790246 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.790250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.790251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.790332 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:12.790624 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:12.790629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:12.790630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:12.790810 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:12.790935 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:12.791193 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 16:07:12.791308 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:12.791481 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:12.783574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51422)
0924 16:07:12.783989 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:12.784006 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:12.784140 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:12.784425 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:12.784427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:12.784582 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:12.784667 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.784672 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.785726 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.785729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.786366 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:12.786376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:12.786841 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:12.786845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:12.786929 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:12.787527 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:12.787545 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:12.788001 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:12.789918 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.789921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.789923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.790123 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.790433 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:12.790437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:12.790439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:12.790524 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:12.790925 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:12.790928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:12.791031 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 16:07:12.791140 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 16:07:12.791298 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":88,"threads_started":1}
I20260924 16:07:12.792382 19508 tablet_copy_service.cc:140] P bb4abb37975e499395cd083ddcfcd578: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6e60b5d591c7447fb748dd55655c0a12 ({username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:51422)
I20260924 16:07:12.792588 19508 tablet_copy_service.cc:161] P bb4abb37975e499395cd083ddcfcd578: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6e60b5d591c7447fb748dd55655c0a12 at {username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:51422: session id = 6e60b5d591c7447fb748dd55655c0a12-00000000000000000000000000000000
I20260924 16:07:12.794023 19508 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 16:07:12.795603 19952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 16:07:12.797442 19952 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: tablet copy: Starting download of 0 data blocks...
I20260924 16:07:12.797645 19952 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: tablet copy: Starting download of 1 WAL segments...
I20260924 16:07:12.799284 19952 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 16:07:12.800812 19508 tablet_copy_service.cc:342] P bb4abb37975e499395cd083ddcfcd578: Request end of tablet copy session 6e60b5d591c7447fb748dd55655c0a12-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:51422
I20260924 16:07:12.800930 19508 tablet_copy_service.cc:433] P bb4abb37975e499395cd083ddcfcd578: ending tablet copy session 6e60b5d591c7447fb748dd55655c0a12-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6e60b5d591c7447fb748dd55655c0a12
I20260924 16:07:12.807608 19952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:12.809221 20062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.809224 20065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.809796 19952 server_base.cc:1034] running on GCE node
W20260924 16:07:12.810372 20063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.810596 19952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:12.810899 19952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:12.812055 19952 hybrid_clock.cc:648] HybridClock initialized: now 1790266032812045 us; error 41 us; skew 500 ppm
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:12.813484 19952 init.cc:377] Logged in from keytab as oryx/127.13.8.125@KRBTEST.COM (short username oryx)
I20260924 16:07:12.814003 19952 webserver.cc:460] Webserver started at http://127.13.8.125:33353/ using document root <none> and password file <none>
I20260924 16:07:12.814189 19952 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.814254 19952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.815330 19952 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:12.816236 20072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.816483 19952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:12.816577 19952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "6e60b5d591c7447fb748dd55655c0a12"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "2ba4958873040fe35f77b7d8fd79d394"
server_key_iv: "c7799212e8817b53132d37e1ef99ee88"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.816680 19952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.838961 19952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:12.840993 19952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:12.846606 19952 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.125:40327
I20260924 16:07:12.847205 19952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
I20260924 16:07:12.847635 20125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.125:40327 every 8 connection(s)
I20260924 16:07:12.848845 20126 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:07:12.849625 20126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Bootstrap starting.
I20260924 16:07:12.857700 20126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:07:12.857993 20126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Bootstrap complete.
I20260924 16:07:12.858518 20126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } }
I20260924 16:07:12.858649 20126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6e60b5d591c7447fb748dd55655c0a12, State: Initialized, Role: LEARNER
I20260924 16:07:12.858763 20126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } }
I20260924 16:07:12.859118 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } }
I20260924 16:07:12.859277 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: LEARNER
I20260924 16:07:12.859292 20126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:12.862097 20140 catalog_manager.cc:1259] Loaded cluster ID: ff812a99ca1b471c8013a8d455d27070
I20260924 16:07:12.862195 20140 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: loading cluster ID for follower catalog manager: success
I20260924 16:07:12.863833 20140 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: acquiring CA information for follower catalog manager: success
I20260924 16:07:12.864928 20140 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 16:07:12.935672 20059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:12.935949 20059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:12.936036 20059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:12.940265 20059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:12.940331 20059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:12.940351 20059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:12.940411 20059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:12.940440 20059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:12.945999 20059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.124.keytab
--principal=oryx/127.13.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:38309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:12.947293 20059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:12.948381 20059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:12.957333 20145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:12.957366 20148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.957343 20059 server_base.cc:1034] running on GCE node
W20260924 16:07:12.957348 20146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:12.958103 20059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:12.959304 20059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:12.960429 20059 hybrid_clock.cc:648] HybridClock initialized: now 1790266032960402 us; error 37 us; skew 500 ppm
Sep 24 16:07:12 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:12.963150 20059 init.cc:377] Logged in from keytab as oryx/127.13.8.124@KRBTEST.COM (short username oryx)
I20260924 16:07:12.964059 20059 webserver.cc:460] Webserver started at http://127.13.8.124:41221/ using document root <none> and password file <none>
I20260924 16:07:12.964658 20059 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:12.964753 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:12.964993 20059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:12.967164 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/instance:
uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "54342799154105e5c3dfc9cc5142e939"
server_key_iv: "337a05d1a64d6b0fee966f7ed0ca12ef"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.967761 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal/instance:
uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "54342799154105e5c3dfc9cc5142e939"
server_key_iv: "337a05d1a64d6b0fee966f7ed0ca12ef"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.971556 20059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:07:12.974013 20155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:12.975395 20059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:12.975546 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "54342799154105e5c3dfc9cc5142e939"
server_key_iv: "337a05d1a64d6b0fee966f7ed0ca12ef"
server_key_version: "encryptionkey@0"
I20260924 16:07:12.975665 20059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:12.989346 20059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:12.992264 20059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:12.992462 20059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:12.999274 20059 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.124:38309
I20260924 16:07:12.999540 20208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.124:38309 every 8 connection(s)
I20260924 16:07:13.000268 20059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
I20260924 16:07:13.001101 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20059
I20260924 16:07:13.003316 20209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:13.007884 20209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
I20260924 16:07:13.015758 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.002581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.13.8.124:38309)
0924 16:07:13.003029 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.003047 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.003094 (+ 47us) server_negotiation.cc:407] Connection header received
0924 16:07:13.003836 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.003854 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.004169 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.004482 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.005395 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.006462 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.007340 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.007819 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.007998 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.010879 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.010887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.010904 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.010941 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.012669 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.013137 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.013142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.013146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.013250 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.013547 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.013550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.013552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.014060 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.014320 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.014603 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.014900 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":196,"threads_started":1}
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.124@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:13.019606 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.009825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51426)
0924 16:07:13.010460 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.010508 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.010655 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.010936 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.010942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.011468 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.011608 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.011614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.012509 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.012513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.013386 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.013399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.013985 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.013988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.014061 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.014627 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.014641 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.016327 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.018274 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.018282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.018294 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.018500 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.018749 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.018751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.018752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.018837 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.019165 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.019169 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.019264 (+ 95us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.019361 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.019480 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":89,"threads_started":1}
I20260924 16:07:13.019625 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.009966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.13.8.126:39979)
0924 16:07:13.010082 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.010086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.010598 (+ 512us) server_negotiation.cc:407] Connection header received
0924 16:07:13.010766 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.010769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.010815 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.010907 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.011725 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.012376 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.013527 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.013869 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.013981 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.016465 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.016469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.016471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.016498 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.018131 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.018589 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.018591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.018593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.018667 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.018917 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.018919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.018920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.019084 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.019155 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.019433 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.019519 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:07:13.020725 20209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.124@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
I20260924 16:07:13.028267 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.021148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36962)
0924 16:07:13.021259 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.021275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.021355 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.021700 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.021702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.021818 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.021919 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.021924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.022828 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.022831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.023379 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.023387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.023819 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.023821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.023888 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.024243 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.024254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.025394 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.026968 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.026971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.026972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.027191 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.027446 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.027448 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.027449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.027539 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.027885 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.027888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.027962 (+ 74us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.028049 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.028158 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 16:07:13.028353 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.021270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.13.8.125:40327)
0924 16:07:13.021508 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.021518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.021530 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:13.021564 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.021566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.021603 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.021673 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.022013 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.022713 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.023484 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.023728 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.023799 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.025516 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.025519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.025521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.025543 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.026850 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.027289 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.027292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.027293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.027366 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.027625 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.027627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.027628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.027782 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.027854 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.028102 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.028195 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260924 16:07:13.029039 20209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:07:13.031551 19679 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266032, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.124@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
I20260924 16:07:13.037137 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.029495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37248)
0924 16:07:13.029602 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.029616 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.029673 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.029848 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.029850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.029964 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.030026 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.030030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.031203 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.031208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.031985 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.031995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.032560 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.032562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.032618 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.033059 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.033072 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.034134 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.035841 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.035846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.035848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.036044 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.036302 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.036306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.036307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.036394 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.036720 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.036722 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.036790 (+ 68us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.036872 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.037035 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 16:07:13.037251 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.029545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.13.8.124:38309)
0924 16:07:13.029647 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.029650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.029660 (+ 10us) server_negotiation.cc:407] Connection header received
0924 16:07:13.029730 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.029732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.029763 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.029821 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.030114 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.031083 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.032121 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.032436 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.032557 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.034308 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.034311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.034313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.034333 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.035711 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.036128 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.036131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.036132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.036202 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.036470 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.036472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.036473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.036640 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.036711 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.036937 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.037090 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":38}
I20260924 16:07:13.040017 20209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Bootstrap starting.
I20260924 16:07:13.042425 20209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:13.043190 20209 log.cc:824] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:13.045037 20209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: No bootstrap required, opened a new log
I20260924 16:07:13.048198 20209 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } }
I20260924 16:07:13.048385 20209 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:13.048463 20209 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a8a0943bd34f38b8beb0a07827b55b, State: Initialized, Role: FOLLOWER
I20260924 16:07:13.048977 20209 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } }
I20260924 16:07:13.049455 20216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } } }
I20260924 16:07:13.049583 20216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:13.050159 20209 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: configured and running, proceeding with master startup.
W20260924 16:07:13.055120 20227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:07:13.055197 20227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:07:13.058363 20059 master_runner.cc:428] Detected that this master d6a8a0943bd34f38b8beb0a07827b55b is joining an existing cluster
I20260924 16:07:13.058411 20059 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.8.124:38309
I20260924 16:07:13.069235 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19550
I20260924 16:07:13.071573 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.061038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.13.8.125:40327)
0924 16:07:13.061158 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.061163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.062443 (+ 1280us) server_negotiation.cc:407] Connection header received
0924 16:07:13.062595 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.062598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.062642 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.062690 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.063379 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.064153 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.065860 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.066136 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.066282 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.067368 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.067371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.067373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.067395 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.069047 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.069915 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.069922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.069927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.070016 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.070350 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.070356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.070361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.070594 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.070670 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.071126 (+ 456us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.071279 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:13.072154 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.060983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36966)
0924 16:07:13.062372 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.062383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.062500 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.062819 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.062822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.062988 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.063274 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.063283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.064861 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.064865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.065730 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.065741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.066273 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.066279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.066370 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.066763 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.066776 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.067224 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.069147 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.069151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.069154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.069800 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.070127 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.070133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.070138 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.070249 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.070802 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.070807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.070925 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.071783 (+ 858us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.071987 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1319,"mutex_wait_us":125,"thread_start_us":73,"threads_started":1}
I20260924 16:07:13.077790 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:41097
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:39979
--webserver_port=36513
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:07:13.082337 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.060589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37250)
0924 16:07:13.061248 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.061260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.061345 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.061650 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.061652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.061952 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.062076 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.062084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.063007 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.063010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.064806 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.064814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.065401 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.065404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.065520 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.066350 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.066368 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.078618 (+ 12250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.080872 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.080876 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.080878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.081114 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.081356 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.081361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.081430 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.081821 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.081824 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.081902 (+ 78us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.081987 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.082128 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":1014,"thread_start_us":108,"threads_started":1}
I20260924 16:07:13.083189 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.060544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51436)
0924 16:07:13.060938 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.060952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.061051 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.061636 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.061640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.061914 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.062030 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.062035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.063007 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.063010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.064120 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.064142 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.064767 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.064770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.064869 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.065403 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.065416 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.078671 (+ 13255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.081121 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.081132 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.081396 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.081736 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.081747 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.081850 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.082387 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.082392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.082502 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.082628 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.083007 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":273,"thread_start_us":112,"threads_started":1}
I20260924 16:07:13.083379 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.061328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.13.8.126:39979)
0924 16:07:13.061437 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.061440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.061452 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:13.061494 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.061497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.061534 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.061621 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.062148 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.062908 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.064262 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.064620 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.064742 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.078882 (+ 14140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.078889 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.078895 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.078931 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.080984 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081529 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.081535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.081540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.081625 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081953 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.081958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.081963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.082204 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.082338 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.083011 (+ 673us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.083154 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 16:07:13.083498 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.060692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.13.8.124:38309)
0924 16:07:13.060810 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.060814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.061319 (+ 505us) server_negotiation.cc:407] Connection header received
0924 16:07:13.061436 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.061439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.061481 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.061527 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.062194 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.062897 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.064915 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.065183 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.065246 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.078764 (+ 13518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.078768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.078771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.078812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.080759 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081207 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.081210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.081212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.081278 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.081517 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.081520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.081521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.081713 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.081792 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.082647 (+ 855us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.083376 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":54}
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266031, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.126@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
I20260924 16:07:13.096972 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.087441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37260)
0924 16:07:13.087608 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.087627 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.087724 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.088018 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.088023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.088264 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.088405 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.088416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.089439 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.089444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.090453 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.090469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.091072 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.091079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.091220 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.091740 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.091754 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.093271 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.095374 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.095377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.095380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.095617 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.095926 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.095929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.095931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.096020 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.096441 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.096444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.096541 (+ 97us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.096645 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.096794 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260924 16:07:13.096979 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.087666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.13.8.124:38309)
0924 16:07:13.087762 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.087766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.087778 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:13.087845 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.087847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.087895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.087976 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.088545 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.089317 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.090600 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.090947 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.091054 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.093407 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.093411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.093414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.093441 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.095258 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.095732 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.095735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.095737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.095821 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.096114 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.096117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.096119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.096345 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.096440 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.096717 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.096827 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 16:07:13.098287 19488 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.8.124:38309: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
member_type: NON_VOTER
last_known_addr {
host: "127.13.8.124"
port: 38309
}
attrs {
promote: true
}
}
dest_uuid: "bb4abb37975e499395cd083ddcfcd578"
cas_config_opid_index: 5
I20260924 16:07:13.098920 19488 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } }
I20260924 16:07:13.099941 20049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.101395 20049 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.8.124:38309
I20260924 16:07:13.101562 20049 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.101662 20049 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.102607 20059 master.cc:537] Master@127.13.8.124:38309 shutting down...
I20260924 16:07:13.102860 20183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:13.104355 20183 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 16:07:13.105875 20048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 16:07:13.107606 19475 consensus_peers.cc:487] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 -> Peer d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124:38309): Couldn't send request to peer d6a8a0943bd34f38b8beb0a07827b55b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 16:07:13.111076 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.102092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.13.8.125:40327)
0924 16:07:13.102226 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.102257 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.106428 (+ 4171us) server_negotiation.cc:407] Connection header received
0924 16:07:13.106621 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.106624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.106687 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.106754 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.107260 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.108094 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.109705 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.110394 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.110500 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.110655 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.110848 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.110937 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:13.111083 20059 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:13.111441 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.101939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:36982)
0924 16:07:13.106348 (+ 4409us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.106364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.106501 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.107028 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.107032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.107048 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.107135 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.107140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.108303 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.108307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.109562 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.109572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.110534 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.110541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.110644 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.110700 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.111263 (+ 563us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.111337 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4327}
I20260924 16:07:13.111699 20059 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:13.111845 20059 tablet_replica.cc:330] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: stopping tablet replica
I20260924 16:07:13.111936 20099 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 LEARNER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 16:07:13.112342 20048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 16:07:13.114106 20099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.116356 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4abb37975e499395cd083ddcfcd578. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.116475 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: LEARNER
I20260924 16:07:13.116568 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: NON_VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: true } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.116644 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: LEARNER
I20260924 16:07:13.117470 20049 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: attempting to promote NON_VOTER 6e60b5d591c7447fb748dd55655c0a12 to VOTER
I20260924 16:07:13.117702 20059 master.cc:552] Master@127.13.8.124:38309 shutdown complete.
I20260924 16:07:13.117753 20059 master_runner.cc:305] Clearing existing system tablet
I20260924 16:07:13.117909 20049 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } }
I20260924 16:07:13.117997 20059 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:13.118045 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:13.119035 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.118756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37266)
0924 16:07:13.118882 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.118954 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 16:07:13.119374 20099 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 LEARNER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 16:07:13.119736 20048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 16:07:13.119853 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.119626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37276)
0924 16:07:13.119749 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.119793 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260924 16:07:13.119973 19475 consensus_peers.cc:487] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 -> Peer d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124:38309): Couldn't send request to peer d6a8a0943bd34f38b8beb0a07827b55b. Status: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 16:07:13.121256 20048 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6e60b5d591c7447fb748dd55655c0a12 (127.13.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.121580 20099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6e60b5d591c7447fb748dd55655c0a12 (127.13.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.122869 20048 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.122990 20048 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.123126 20048 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.123234 20048 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.123184 20238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.123298 20238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:13.123637 20059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:07:13.124588 20241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.124810 20059 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:13.125082 20059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:13.125161 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "54342799154105e5c3dfc9cc5142e939"
server_key_iv: "337a05d1a64d6b0fee966f7ed0ca12ef"
server_key_version: "encryptionkey@0"
I20260924 16:07:13.127542 20059 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:13.129945 20059 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 16:07:13.130095 20059 log.cc:1196] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 16:07:13.130424 20059 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Deleting consensus metadata
I20260924 16:07:13.130632 20059 master_runner.cc:315] Copying system tablet from 127.13.8.126:39979
I20260924 16:07:13.131810 20059 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: tablet copy: Beginning tablet copy session from remote peer at address 127.13.8.126:39979
I20260924 16:07:13.141075 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.132365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51446)
0924 16:07:13.132739 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.132754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.132861 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.133115 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.133118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.133330 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.133431 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.133437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.134550 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.134554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.135539 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.135550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.136210 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.136214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.136329 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.136887 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.136902 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.137343 (+ 441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.139349 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.139352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.139354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.139607 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.139915 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.139918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.139920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.140009 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.140496 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.140499 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.140596 (+ 97us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.140702 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.140837 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260924 16:07:13.141464 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.132483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.13.8.126:39979)
0924 16:07:13.132598 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.132602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.132841 (+ 239us) server_negotiation.cc:407] Connection header received
0924 16:07:13.132939 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.132942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.132992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.133084 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.133553 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.134417 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.135665 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.136003 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.136125 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.137484 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.137488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.137491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.137516 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.139235 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.139709 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.139713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.139715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.139810 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.140113 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.140116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.140117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.140386 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.140505 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.140793 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.140933 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:13.141983 19508 tablet_copy_service.cc:140] P bb4abb37975e499395cd083ddcfcd578: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d6a8a0943bd34f38b8beb0a07827b55b ({username='oryx', principal='oryx/127.13.8.124@KRBTEST.COM'} at 127.0.0.1:51446)
I20260924 16:07:13.142067 19508 tablet_copy_service.cc:161] P bb4abb37975e499395cd083ddcfcd578: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d6a8a0943bd34f38b8beb0a07827b55b at {username='oryx', principal='oryx/127.13.8.124@KRBTEST.COM'} at 127.0.0.1:51446: session id = d6a8a0943bd34f38b8beb0a07827b55b-00000000000000000000000000000000
I20260924 16:07:13.143127 19508 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 16:07:13.145393 20059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 16:07:13.147778 20059 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: tablet copy: Starting download of 0 data blocks...
I20260924 16:07:13.147964 20059 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: tablet copy: Starting download of 1 WAL segments...
I20260924 16:07:13.149469 20059 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 16:07:13.151387 19508 tablet_copy_service.cc:342] P bb4abb37975e499395cd083ddcfcd578: Request end of tablet copy session d6a8a0943bd34f38b8beb0a07827b55b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.8.124@KRBTEST.COM'} at 127.0.0.1:51446
I20260924 16:07:13.151459 19508 tablet_copy_service.cc:433] P bb4abb37975e499395cd083ddcfcd578: ending tablet copy session d6a8a0943bd34f38b8beb0a07827b55b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d6a8a0943bd34f38b8beb0a07827b55b
I20260924 16:07:13.158773 20059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:13.160458 20248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.160473 20251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.161052 20249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:13.162678 20059 server_base.cc:1034] running on GCE node
I20260924 16:07:13.162823 20059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:13.163118 20059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:13.164270 20059 hybrid_clock.cc:648] HybridClock initialized: now 1790266033164256 us; error 27 us; skew 500 ppm
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:13.165860 20059 init.cc:377] Logged in from keytab as oryx/127.13.8.124@KRBTEST.COM (short username oryx)
I20260924 16:07:13.166345 20059 webserver.cc:460] Webserver started at http://127.13.8.124:46507/ using document root <none> and password file <none>
I20260924 16:07:13.166529 20059 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:13.166571 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:13.167785 20059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 16:07:13.168779 20259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.169006 20059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 16:07:13.169085 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "54342799154105e5c3dfc9cc5142e939"
server_key_iv: "337a05d1a64d6b0fee966f7ed0ca12ef"
server_key_version: "encryptionkey@0"
I20260924 16:07:13.169162 20059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:13.176139 20059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:13.178299 20059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:13.183103 20059 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.124:38309
I20260924 16:07:13.183667 20311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.124:38309 every 8 connection(s)
I20260924 16:07:13.183760 20059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
I20260924 16:07:13.185210 20312 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:07:13.186056 20312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Bootstrap starting.
I20260924 16:07:13.194831 20312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:07:13.195101 20312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: Bootstrap complete.
I20260924 16:07:13.195570 20312 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } }
I20260924 16:07:13.195700 20312 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6a8a0943bd34f38b8beb0a07827b55b, State: Initialized, Role: LEARNER
I20260924 16:07:13.195819 20312 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } }
I20260924 16:07:13.196151 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.196328 20312 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:13.196336 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: LEARNER
I20260924 16:07:13.199795 20326 catalog_manager.cc:1259] Loaded cluster ID: ff812a99ca1b471c8013a8d455d27070
I20260924 16:07:13.199864 20326 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: loading cluster ID for follower catalog manager: success
I20260924 16:07:13.201382 20326 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: acquiring CA information for follower catalog manager: success
I20260924 16:07:13.202656 20326 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 16:07:13.236395 20236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:13.236735 20236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:13.236797 20236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:13.242820 20236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:13.242887 20236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:13.242970 20236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:13.250854 20236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=36513
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:13.251974 20236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:13.253149 20236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:07:13.261173 19813 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
W20260924 16:07:13.264261 20331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.264279 20334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:13.264317 20236 server_base.cc:1034] running on GCE node
W20260924 16:07:13.264297 20332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:13.264832 20236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:13.265419 20236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:13.266577 20236 hybrid_clock.cc:648] HybridClock initialized: now 1790266033266553 us; error 31 us; skew 500 ppm
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:13.269233 20236 init.cc:377] Logged in from keytab as oryx/127.13.8.65@KRBTEST.COM (short username oryx)
I20260924 16:07:13.270045 20236 webserver.cc:460] Webserver started at http://127.13.8.65:36513/ using document root <none> and password file <none>
I20260924 16:07:13.270697 20236 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:13.270812 20236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:13.274545 20236 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260924 16:07:13.276799 20341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.278024 20236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:13.278168 20236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "e85af61ce1e240c9aa23fe66d1f328aa"
format_stamp: "Formatted at 2026-09-24 16:07:11 on dist-test-slave-vj6n"
server_key: "d6effcb6235000f91e409360f3bb4f7e"
server_key_iv: "67bf1c6ec83b82975d623dd985fd4a23"
server_key_version: "encryptionkey@0"
I20260924 16:07:13.278761 20236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:13.288664 20236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:13.291661 20236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:13.291862 20236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:13.292433 20236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:13.293291 20236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:13.293360 20236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.293421 20236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:13.293474 20236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.304021 20236 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:41097
I20260924 16:07:13.304047 20455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:41097 every 8 connection(s)
I20260924 16:07:13.305009 20236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:13.309510 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20236
I20260924 16:07:13.309741 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19684
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:13.318436 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:39245
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:39979
--webserver_port=33623
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:07:13.322945 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.308406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:46971 (local address 127.13.8.126:39979)
0924 16:07:13.308523 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.308527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.309340 (+ 813us) server_negotiation.cc:407] Connection header received
0924 16:07:13.309526 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.309532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.309586 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.309644 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.310974 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.311751 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.315221 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.315652 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.315766 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.318624 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.318629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.318633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.318665 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.321089 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321594 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.321600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.321603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.321685 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321995 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.321999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.322002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.322202 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.322333 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.322651 (+ 318us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.322774 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:07:13.322949 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.307023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:51407 (local address 127.13.8.124:38309)
0924 16:07:13.307316 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.307322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.308359 (+ 1037us) server_negotiation.cc:407] Connection header received
0924 16:07:13.309015 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.309018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.309065 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.309156 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.310906 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.311985 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.313832 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.314087 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.314203 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.318767 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.318775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.318778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.318808 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.320727 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321227 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.321230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.321232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.321318 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321640 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.321646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.321649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.321874 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.321948 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.322651 (+ 703us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.322774 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260924 16:07:13.323526 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.306796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59507 (local address 127.13.8.125:40327)
0924 16:07:13.307047 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.307052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.307947 (+ 895us) server_negotiation.cc:407] Connection header received
0924 16:07:13.309307 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.309312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.309365 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.309443 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.311619 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.312514 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.314031 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.314476 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.314581 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.318285 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.318289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.318291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.318319 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.320314 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.320981 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.320986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.320989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.321091 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321524 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.321531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.321535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.321798 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.321893 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.322691 (+ 798us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.322823 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260924 16:07:13.324601 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.308297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:46971)
0924 16:07:13.309256 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.309269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.309412 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.310556 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.310559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.310734 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.310841 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.310848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.311926 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.311929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.315093 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.315106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.315901 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.315907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.316030 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.316541 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.316562 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.318476 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.321217 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.321221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.321485 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.321809 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.321814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.321892 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.322386 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.322389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.322471 (+ 82us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.323157 (+ 686us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.323559 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":2103,"thread_start_us":81,"threads_started":1}
I20260924 16:07:13.324919 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.307282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.13.8.65:51407)
0924 16:07:13.308276 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.308290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.308789 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.309373 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.309383 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.310440 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.310735 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.310755 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.312127 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.312129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.313712 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.313726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.314280 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.314285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.314471 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.315367 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.315399 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.318375 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.320856 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.320862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.320866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.321103 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.321440 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.321450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.321539 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.322019 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.322022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.322373 (+ 351us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.323157 (+ 784us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.323603 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":220,"spinlock_wait_cycles":15232,"thread_start_us":134,"threads_started":1}
I20260924 16:07:13.324623 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.307282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.13.8.65:59507)
0924 16:07:13.307783 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.307833 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.308789 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.309575 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.309578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.309978 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.310738 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.310756 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.312650 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.312656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.313887 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.313901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.314711 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.314717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.314890 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.315408 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.315419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.318024 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.320456 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.320464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.320481 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.320857 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.321205 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.321210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.321211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.321413 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.321952 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.321961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.322376 (+ 415us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.323871 (+ 1495us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.324038 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":30,"spinlock_wait_cycles":8576,"thread_start_us":570,"threads_started":1}
I20260924 16:07:13.325968 20456 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:13.326191 20456 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.326294 20458 heartbeater.cc:347] Connected to a master server at 127.13.8.125:40327
I20260924 16:07:13.326361 20458 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.326678 20457 heartbeater.cc:347] Connected to a master server at 127.13.8.124:38309
I20260924 16:07:13.326740 20457 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.326861 20458 heartbeater.cc:511] Master 127.13.8.125:40327 requested a full tablet report, sending...
I20260924 16:07:13.326876 20457 heartbeater.cc:511] Master 127.13.8.124:38309 requested a full tablet report, sending...
I20260924 16:07:13.326859 20456 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
I20260924 16:07:13.328495 19488 ts_manager.cc:194] Re-registered known tserver with Master: e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097)
I20260924 16:07:13.328604 20268 ts_manager.cc:194] Registered new tserver with Master: e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097)
I20260924 16:07:13.328604 20089 ts_manager.cc:194] Registered new tserver with Master: e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097)
I20260924 16:07:13.329391 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:46971
W20260924 16:07:13.432539 20464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:13.432765 20464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:13.432804 20464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:13.436946 20464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:13.437002 20464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:13.437089 20464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:13.442065 20464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:39245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=33623
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:13.443065 20464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:13.443988 20464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:13.453568 20472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.453563 20469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.453563 20470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:13.453613 20464 server_base.cc:1034] running on GCE node
I20260924 16:07:13.454155 20464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:13.454777 20464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:13.455932 20464 hybrid_clock.cc:648] HybridClock initialized: now 1790266033455918 us; error 29 us; skew 500 ppm
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:13.458596 20464 init.cc:377] Logged in from keytab as oryx/127.13.8.66@KRBTEST.COM (short username oryx)
I20260924 16:07:13.459429 20464 webserver.cc:460] Webserver started at http://127.13.8.66:33623/ using document root <none> and password file <none>
I20260924 16:07:13.460073 20464 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:13.460143 20464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:13.464048 20464 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:07:13.466073 20479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.467332 20464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:13.467454 20464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "254277422fba4b1e99b1fd7c7d20b47b"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "ed5bd1b63eb8cdfe398f75815d0e55fa"
server_key_iv: "d1dfac75daba913320798ddc6f3ec98e"
server_key_version: "encryptionkey@0"
I20260924 16:07:13.467944 20464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:13.481711 19947 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
I20260924 16:07:13.494292 20464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:13.497251 20464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:13.497455 20464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:13.498209 20464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:13.499135 20464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:13.499178 20464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.499250 20464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:13.499279 20464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.508273 20464 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39245
I20260924 16:07:13.508311 20593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39245 every 8 connection(s)
I20260924 16:07:13.509265 20464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:13.515640 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20464
I20260924 16:07:13.515816 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19818
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:13.526088 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.513056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:38957 (local address 127.13.8.125:40327)
0924 16:07:13.513195 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.513200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.513315 (+ 115us) server_negotiation.cc:407] Connection header received
0924 16:07:13.513930 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.513934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.513991 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.514094 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.515821 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.516644 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.518672 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.518917 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.519027 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.521653 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.521657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.521659 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.521687 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.523471 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524140 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.524145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.524150 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.524267 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524694 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.524700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.524701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.524936 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.525011 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.525756 (+ 745us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.525878 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:07:13.526063 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.511231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:49001 (local address 127.13.8.124:38309)
0924 16:07:13.511360 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.511365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.512576 (+ 1211us) server_negotiation.cc:407] Connection header received
0924 16:07:13.513958 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.513961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.514052 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.514118 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.515775 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.516609 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.519107 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.519553 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.519645 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.521961 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.521968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.521974 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.522010 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.523963 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524565 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.524571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.524576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.524663 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524978 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.524983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.524988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.525243 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.525322 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.525755 (+ 433us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.525904 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:07:13.526307 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:45627
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:39979
--webserver_port=44547
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:07:13.527954 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.512953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.13.8.66:38957)
0924 16:07:13.513222 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.513235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.513718 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.514313 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.514321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.514809 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.515604 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.515626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.517665 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.517668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.518568 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.518576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.519064 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.519068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.519248 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.519835 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.519853 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.521466 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.523595 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.523601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.523617 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.524016 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.524391 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.524397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.524570 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.525108 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.525118 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.525455 (+ 337us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.526364 (+ 909us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.526750 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":145,"spinlock_wait_cycles":15872,"thread_start_us":81,"threads_started":1}
I20260924 16:07:13.527956 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.511795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.13.8.66:49001)
0924 16:07:13.512387 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.512466 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.513709 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.514370 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.514373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.514785 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.515604 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.515625 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.517817 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.517823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.518947 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.518961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.519811 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.519817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.519947 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.520434 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.520453 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.521811 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.524097 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.524108 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.524453 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.524777 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.524783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.524787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.524878 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.525450 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.525453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.525553 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.526783 (+ 1230us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.526953 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":174,"spinlock_wait_cycles":11648,"thread_start_us":156,"threads_started":1}
I20260924 16:07:13.529215 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.511795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:47927)
0924 16:07:13.512387 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.512466 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.513709 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.514371 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.514373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.514785 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.515604 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.515625 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.516764 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.516771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.518400 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.518411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.523505 (+ 5094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.523511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.523614 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.524102 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.524121 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.525612 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.527586 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.527589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.527591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.527837 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.528155 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.528161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.528166 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.528254 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.528728 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.528731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.528826 (+ 95us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.528924 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.529043 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":204,"spinlock_wait_cycles":93952,"thread_start_us":136,"threads_started":1}
I20260924 16:07:13.529273 20596 heartbeater.cc:347] Connected to a master server at 127.13.8.125:40327
I20260924 16:07:13.529487 20596 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.529767 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.511231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:47927 (local address 127.13.8.126:39979)
0924 16:07:13.511360 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.511365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.512576 (+ 1211us) server_negotiation.cc:407] Connection header received
0924 16:07:13.514050 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.514053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.514110 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.514207 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.515774 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.516602 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.522943 (+ 6341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.523353 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.523477 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.525774 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.525777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.525780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.525805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.527466 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.527952 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.527955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.527957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.528043 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.528360 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.528366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.528371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.528580 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.528670 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.529499 (+ 829us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.529616 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:13.530038 20596 heartbeater.cc:511] Master 127.13.8.125:40327 requested a full tablet report, sending...
I20260924 16:07:13.530622 20595 heartbeater.cc:347] Connected to a master server at 127.13.8.124:38309
I20260924 16:07:13.530714 20595 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.530900 20595 heartbeater.cc:511] Master 127.13.8.124:38309 requested a full tablet report, sending...
I20260924 16:07:13.531136 20594 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:13.531216 20594 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.531363 20594 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
I20260924 16:07:13.531359 20089 ts_manager.cc:194] Registered new tserver with Master: 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:13.531740 20268 ts_manager.cc:194] Registered new tserver with Master: 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:13.531996 19488 ts_manager.cc:194] Re-registered known tserver with Master: 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:13.532613 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:47927
I20260924 16:07:13.631633 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.627816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37290)
0924 16:07:13.628061 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.628075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.628170 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.628481 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.628483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.628497 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.628591 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.628596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.629590 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.629592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.630407 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.630418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.631314 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.631317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.631394 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.631415 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.631459 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.631514 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260924 16:07:13.631683 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.627950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.13.8.124:38309)
0924 16:07:13.628119 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.628126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.628146 (+ 20us) server_negotiation.cc:407] Connection header received
0924 16:07:13.628282 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.628285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.628355 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.628464 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.628697 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.629475 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.630524 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.631190 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.631297 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.631452 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.631488 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.631554 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:07:13.632439 20278 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260924 16:07:13.632830 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4abb37975e499395cd083ddcfcd578. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.632984 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: LEARNER
W20260924 16:07:13.641409 20601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:13.641664 20601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:13.641762 20601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:13.645756 20601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:13.645830 20601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:13.645927 20601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:13.651211 20601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:45627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=44547
--tserver_master_addrs=127.13.8.126:39979,127.13.8.125:40327,127.13.8.124:38309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:13.652267 20601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:13.653246 20601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:13.663587 20610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.663615 20609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:13.663614 20612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:13.663717 20601 server_base.cc:1034] running on GCE node
I20260924 16:07:13.664145 20601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:13.664757 20601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:13.665920 20601 hybrid_clock.cc:648] HybridClock initialized: now 1790266033665892 us; error 38 us; skew 500 ppm
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:13.668614 20601 init.cc:377] Logged in from keytab as oryx/127.13.8.67@KRBTEST.COM (short username oryx)
I20260924 16:07:13.669517 20601 webserver.cc:460] Webserver started at http://127.13.8.67:44547/ using document root <none> and password file <none>
I20260924 16:07:13.670085 20601 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:13.670197 20601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:13.674371 20601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260924 16:07:13.676296 20619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.677477 20601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:13.677608 20601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "ed31894d5d5a4e8baf091ecbb6729e38"
format_stamp: "Formatted at 2026-09-24 16:07:12 on dist-test-slave-vj6n"
server_key: "129f32e71104099ed6a5c74e4a3609d2"
server_key_iv: "8cfa1d8cebc652059688487914ee515a"
server_key_version: "encryptionkey@0"
I20260924 16:07:13.678115 20601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:13.694857 20601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:13.698271 20601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:13.698472 20601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:13.699155 20601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:13.700019 20601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:13.700088 20601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.700160 20601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:13.700219 20601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:13.710155 20601 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:45627
I20260924 16:07:13.710175 20733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:45627 every 8 connection(s)
I20260924 16:07:13.711221 20601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:13.715101 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20601
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.124@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.125@KRBTEST.COM
Sep 24 16:07:13 dist-test-slave-vj6n krb5kdc[19443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266033, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:13.730084 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.714993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46735 (local address 127.13.8.125:40327)
0924 16:07:13.715136 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.715141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.715744 (+ 603us) server_negotiation.cc:407] Connection header received
0924 16:07:13.715846 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.715852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.715910 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.716003 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.717628 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.718439 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720449 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.720794 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720864 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.724599 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.724603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.724605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.724632 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.727263 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.728374 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.728378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.728379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.728463 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.728849 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.728852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.728854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.729044 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.729120 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.729789 (+ 669us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.729901 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:13.731990 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.13.8.125:40327)
0924 16:07:13.722257 (+ 3876us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.722263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.722278 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:13.722320 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.722324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.722378 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.722511 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.723372 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.724245 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.725416 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.725859 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.726394 (+ 535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.727692 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.727698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.727702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.727729 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.729483 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.730004 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.730007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.730009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.730092 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.730444 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.730449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.730454 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.730680 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.731664 (+ 984us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.731739 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.731850 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1816,"thread_start_us":91,"threads_started":1}
I20260924 16:07:13.732039 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.713578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:59677)
0924 16:07:13.714613 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.714627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.715403 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.717433 (+ 2030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.717436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.717650 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.717748 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.717755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.719181 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.719185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.720213 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.720225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.720902 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.721158 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.721941 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.721978 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.724901 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.729700 (+ 4799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.729706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.729708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.729943 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.730282 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.730285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.730287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.730376 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.730815 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.730818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.730916 (+ 98us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.731013 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.731173 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":141,"spinlock_wait_cycles":49280,"thread_start_us":136,"threads_started":1}
I20260924 16:07:13.732448 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.714894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.13.8.67:46735)
0924 16:07:13.715364 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.715378 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.715762 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.716147 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.716153 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.716709 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.717455 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.717478 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.718615 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.718619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.720349 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.720355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.721019 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.721023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.721162 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.721942 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.721967 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.724415 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.727866 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.727876 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.727896 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.728263 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.728573 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.728576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.728578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.728744 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.729205 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.729225 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.729519 (+ 294us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.730530 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.730884 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":395,"spinlock_wait_cycles":11904,"thread_start_us":73,"threads_started":1}
I20260924 16:07:13.732702 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.717036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:59677 (local address 127.13.8.126:39979)
0924 16:07:13.717191 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.717195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.717210 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:13.717251 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.717253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.717305 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.717380 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.717911 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.719037 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720318 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.720656 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720744 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.725044 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.725048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.725050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.725074 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.729081 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.730051 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.730057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.730061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.730157 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.730479 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.730485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.730489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.730671 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.730770 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.732448 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.732563 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":74}
I20260924 16:07:13.733565 20734 heartbeater.cc:347] Connected to a master server at 127.13.8.126:39979
I20260924 16:07:13.733817 20734 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.734110 20736 heartbeater.cc:347] Connected to a master server at 127.13.8.125:40327
I20260924 16:07:13.734193 20736 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.734508 20736 heartbeater.cc:511] Master 127.13.8.125:40327 requested a full tablet report, sending...
I20260924 16:07:13.734606 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.717760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.13.8.126:39979)
0924 16:07:13.718956 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.718960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.718973 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:13.719007 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.719010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.719053 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.719132 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.720618 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.722325 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.724759 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.725446 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.725586 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.727528 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.727534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.727539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.727563 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.732149 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.732660 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.732663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.732665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.732759 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.733083 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.733086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.733088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.733285 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.733422 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.734049 (+ 627us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.734305 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":328,"server-negotiator.queue_time_us":1119,"thread_start_us":63,"threads_started":1}
I20260924 16:07:13.735147 20734 heartbeater.cc:511] Master 127.13.8.126:39979 requested a full tablet report, sending...
I20260924 16:07:13.735795 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.713074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:48521 (local address 127.13.8.124:38309)
0924 16:07:13.713208 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.713213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.714296 (+ 1083us) server_negotiation.cc:407] Connection header received
0924 16:07:13.715648 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.715652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.715719 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.715800 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.717638 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.718398 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720368 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.720701 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720779 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.726091 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.726094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.726097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.726121 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.729587 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.734366 (+ 4779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.734370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.734371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.734458 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.734806 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.734810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.734812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.735007 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.735080 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.735498 (+ 418us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.735646 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:07:13.736053 19488 ts_manager.cc:194] Re-registered known tserver with Master: ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:13.736097 20089 ts_manager.cc:194] Registered new tserver with Master: ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:13.736299 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.713578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.13.8.67:48521)
0924 16:07:13.714104 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.714157 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.715407 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.716131 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.716144 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.716699 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:13.717457 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.717478 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.718575 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.718578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.720229 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.720240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.720948 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.720954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.721121 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.722040 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:13.722055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:13.725948 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:13.733469 (+ 7521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.733473 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.733475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.733731 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.734607 (+ 876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:13.734610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:13.734612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:13.734697 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:13.735162 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:13.735167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:13.735256 (+ 89us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.735970 (+ 714us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.736118 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":321,"spinlock_wait_cycles":1792,"thread_start_us":148,"threads_started":1}
I20260924 16:07:13.736835 19488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:59677
I20260924 16:07:13.736866 20735 heartbeater.cc:347] Connected to a master server at 127.13.8.124:38309
I20260924 16:07:13.736930 20735 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:13.737107 20735 heartbeater.cc:511] Master 127.13.8.124:38309 requested a full tablet report, sending...
I20260924 16:07:13.737726 20268 ts_manager.cc:194] Registered new tserver with Master: ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:13.737613 19488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:13.740741 19488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:13.743165 20602 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: NON_VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: true } } }
I20260924 16:07:13.743320 20602 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.745394 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.717225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.13.8.124:38309)
0924 16:07:13.722721 (+ 5496us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.722726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.722742 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:07:13.722780 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.722784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.723331 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.727457 (+ 4126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:13.728288 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.729041 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.730072 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.730428 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.730518 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.735107 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:13.735110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:13.735113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:13.735137 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:13.743638 (+ 8501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.744132 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.744135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.744137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.744221 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:13.744524 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:13.744527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:13.744529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:13.744734 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:13.744847 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.745133 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.745250 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5420,"thread_start_us":61,"threads_started":1}
I20260924 16:07:13.747606 20602 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: attempting to promote NON_VOTER d6a8a0943bd34f38b8beb0a07827b55b to VOTER
I20260924 16:07:13.747970 20602 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } }
I20260924 16:07:13.749038 20278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 LEARNER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260924 16:07:13.749184 20099 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Refusing update from remote peer bb4abb37975e499395cd083ddcfcd578: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260924 16:07:13.749481 20752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 16:07:13.749812 20753 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 16:07:13.751305 20752 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.751834 20099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.752362 20278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.752813 20752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.752907 20752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.753052 20752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.753154 20752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.753302 20752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.753377 20752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:13.754333 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.754438 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:13.755478 20761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb4abb37975e499395cd083ddcfcd578" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:13.755628 20761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:13.762279 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.756112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45627 (local address 127.0.0.1:41904)
0924 16:07:13.756259 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.756275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.756355 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.758284 (+ 1929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.758287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.758303 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.758444 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.758451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.759790 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.759794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.760749 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.760759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.761820 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.761823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.761910 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.761936 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.762016 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.762085 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":96,"spinlock_wait_cycles":3968}
I20260924 16:07:13.762349 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.756421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.13.8.67:45627)
0924 16:07:13.757829 (+ 1408us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.757839 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.757860 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:07:13.757927 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.757932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.758105 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.758487 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.758623 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.759661 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.760893 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.761701 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.761810 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.761956 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.762042 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.762141 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1314,"thread_start_us":98,"threads_started":1}
I20260924 16:07:13.763010 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.756729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41097 (local address 127.0.0.1:34824)
0924 16:07:13.757472 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.757487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.757574 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.758085 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.758088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.758104 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.758279 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.758289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.759712 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.759716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.761247 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.761258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.762295 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.762299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.762744 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.762770 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.762824 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.762888 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":17,"thread_start_us":108,"threads_started":1}
I20260924 16:07:13.763521 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.756859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39245 (local address 127.0.0.1:47676)
0924 16:07:13.757542 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.757553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.757642 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.758299 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.758302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.758321 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.758564 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.758572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.760366 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.760369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.761560 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.761567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.762675 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.762679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.762937 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.762972 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.763041 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.763402 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":31,"spinlock_wait_cycles":768,"thread_start_us":94,"threads_started":1}
I20260924 16:07:13.763878 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.757116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.13.8.65:41097)
0924 16:07:13.757573 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.757587 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.757613 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:07:13.757699 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.757706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.757938 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.758111 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.758370 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.759545 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.761357 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.762133 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.763441 (+ 1308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.763590 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.763652 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.763747 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":125,"threads_started":1}
I20260924 16:07:13.764010 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.757029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.13.8.66:39245)
0924 16:07:13.757849 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.757857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.757876 (+ 19us) server_negotiation.cc:407] Connection header received
0924 16:07:13.757936 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.757940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.758130 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.758459 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.758651 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.759568 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.761669 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.762548 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.762673 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.762810 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.763780 (+ 970us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.763877 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":90,"threads_started":1}
I20260924 16:07:13.765712 20667 tablet_service.cc:1467] Processing CreateTablet for tablet fa21c108a1d04466a317b9eb52eae382 (DEFAULT_TABLE table=test-table [id=cc5672ee85394aa89438f0496c40288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:13.766414 20527 tablet_service.cc:1467] Processing CreateTablet for tablet fa21c108a1d04466a317b9eb52eae382 (DEFAULT_TABLE table=test-table [id=cc5672ee85394aa89438f0496c40288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:13.766987 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa21c108a1d04466a317b9eb52eae382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:13.766744 20389 tablet_service.cc:1467] Processing CreateTablet for tablet fa21c108a1d04466a317b9eb52eae382 (DEFAULT_TABLE table=test-table [id=cc5672ee85394aa89438f0496c40288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:13.767506 20527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa21c108a1d04466a317b9eb52eae382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:13.767787 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa21c108a1d04466a317b9eb52eae382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:13.775733 20769 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Bootstrap starting.
I20260924 16:07:13.775799 20768 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Bootstrap starting.
I20260924 16:07:13.776837 20767 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Bootstrap starting.
I20260924 16:07:13.777879 20769 tablet_bootstrap.cc:654] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:13.778002 20768 tablet_bootstrap.cc:654] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:13.778733 20769 log.cc:824] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:13.778762 20767 tablet_bootstrap.cc:654] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:13.778836 20768 log.cc:824] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:13.779510 20767 log.cc:824] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:13.780674 20768 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: No bootstrap required, opened a new log
I20260924 16:07:13.780727 20769 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: No bootstrap required, opened a new log
I20260924 16:07:13.780889 20768 ts_tablet_manager.cc:1397] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260924 16:07:13.780900 20769 ts_tablet_manager.cc:1397] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260924 16:07:13.781319 20767 tablet_bootstrap.cc:492] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: No bootstrap required, opened a new log
I20260924 16:07:13.781483 20767 ts_tablet_manager.cc:1397] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260924 16:07:13.784773 20768 raft_consensus.cc:348] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.785009 20768 raft_consensus.cc:374] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:13.785069 20768 raft_consensus.cc:729] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed31894d5d5a4e8baf091ecbb6729e38, State: Initialized, Role: FOLLOWER
I20260924 16:07:13.785533 20768 consensus_queue.cc:260] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.785463 20769 raft_consensus.cc:348] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.785795 20769 raft_consensus.cc:374] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:13.785871 20769 raft_consensus.cc:729] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85af61ce1e240c9aa23fe66d1f328aa, State: Initialized, Role: FOLLOWER
I20260924 16:07:13.786274 20734 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
I20260924 16:07:13.786360 20769 consensus_queue.cc:260] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.786572 20768 ts_tablet_manager.cc:1428] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260924 16:07:13.786916 20767 raft_consensus.cc:348] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.787132 20767 raft_consensus.cc:374] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:13.787186 20767 raft_consensus.cc:729] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254277422fba4b1e99b1fd7c7d20b47b, State: Initialized, Role: FOLLOWER
I20260924 16:07:13.787310 20769 ts_tablet_manager.cc:1428] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260924 16:07:13.787642 20456 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
I20260924 16:07:13.787573 20767 consensus_queue.cc:260] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.790508 20594 heartbeater.cc:503] Master 127.13.8.126:39979 was elected leader, sending a full tablet report...
I20260924 16:07:13.792902 20767 ts_tablet_manager.cc:1428] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
W20260924 16:07:13.806548 20459 tablet.cc:2369] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:13.938130 20773 raft_consensus.cc:482] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:13.938447 20773 raft_consensus.cc:504] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.939601 20773 leader_election.cc:288] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097), 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:13.944686 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.939982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46227 (local address 127.13.8.65:41097)
0924 16:07:13.940120 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.940125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.940145 (+ 20us) server_negotiation.cc:407] Connection header received
0924 16:07:13.940220 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.940227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.940315 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.940410 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.940849 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.941650 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.943400 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.944003 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.944126 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.944188 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.944491 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.944561 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 16:07:13.944686 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.939982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:50269 (local address 127.13.8.66:39245)
0924 16:07:13.940120 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.940125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.940179 (+ 54us) server_negotiation.cc:407] Connection header received
0924 16:07:13.940328 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.940333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.940389 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.940463 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:13.940848 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.941650 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.943400 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.944003 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.944126 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.944188 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:13.944491 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 16:07:13.944561 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:07:13.944712 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.939867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41097 (local address 127.13.8.67:46227)
0924 16:07:13.939975 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.939991 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.940099 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.940450 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.940456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.940477 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.940710 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.940718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.941774 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.941778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.943260 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.943267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.944144 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.944148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.944322 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.944380 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.944456 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.944556 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":340,"spinlock_wait_cycles":8192}
I20260924 16:07:13.944726 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.939867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39245 (local address 127.13.8.67:50269)
0924 16:07:13.940122 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:13.940136 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:13.940207 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:13.940480 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:13.940482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:13.940493 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:13.940724 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.940730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.941774 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.941778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.943254 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:13.943267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:13.944144 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:13.944148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:13.944338 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:13.944387 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 16:07:13.944456 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 16:07:13.944556 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":306}
I20260924 16:07:13.945479 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa21c108a1d04466a317b9eb52eae382" candidate_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "254277422fba4b1e99b1fd7c7d20b47b" is_pre_election: true
I20260924 16:07:13.945479 20410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa21c108a1d04466a317b9eb52eae382" candidate_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" is_pre_election: true
I20260924 16:07:13.945788 20548 raft_consensus.cc:2454] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed31894d5d5a4e8baf091ecbb6729e38 in term 0.
I20260924 16:07:13.945788 20410 raft_consensus.cc:2454] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed31894d5d5a4e8baf091ecbb6729e38 in term 0.
I20260924 16:07:13.946306 20621 leader_election.cc:302] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 254277422fba4b1e99b1fd7c7d20b47b, e85af61ce1e240c9aa23fe66d1f328aa, ed31894d5d5a4e8baf091ecbb6729e38; no voters:
I20260924 16:07:13.946609 20773 raft_consensus.cc:2781] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:13.946688 20773 raft_consensus.cc:482] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:13.946717 20773 raft_consensus.cc:3037] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:13.947808 20773 raft_consensus.cc:504] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.948191 20773 leader_election.cc:288] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [CANDIDATE]: Term 1 election: Requested vote from peers e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097), 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245)
I20260924 16:07:13.948519 20410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa21c108a1d04466a317b9eb52eae382" candidate_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e85af61ce1e240c9aa23fe66d1f328aa"
I20260924 16:07:13.948648 20410 raft_consensus.cc:3037] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:13.948629 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa21c108a1d04466a317b9eb52eae382" candidate_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "254277422fba4b1e99b1fd7c7d20b47b"
I20260924 16:07:13.948792 20548 raft_consensus.cc:3037] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:13.949793 20410 raft_consensus.cc:2454] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed31894d5d5a4e8baf091ecbb6729e38 in term 1.
I20260924 16:07:13.950037 20548 raft_consensus.cc:2454] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed31894d5d5a4e8baf091ecbb6729e38 in term 1.
I20260924 16:07:13.950085 20623 leader_election.cc:302] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e85af61ce1e240c9aa23fe66d1f328aa, ed31894d5d5a4e8baf091ecbb6729e38; no voters:
I20260924 16:07:13.950253 20773 raft_consensus.cc:2781] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:13.950476 20773 raft_consensus.cc:686] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 LEADER]: Becoming Leader. State: Replica: ed31894d5d5a4e8baf091ecbb6729e38, State: Running, Role: LEADER
I20260924 16:07:13.950862 20773 consensus_queue.cc:237] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } }
I20260924 16:07:13.954564 19487 catalog_manager.cc:5495] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } health_report { overall_health: UNKNOWN } } }
W20260924 16:07:13.965694 20737 tablet.cc:2369] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:14.003156 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:13.999336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.13.8.67:45627)
0924 16:07:13.999475 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:13.999482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:13.999497 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:13.999721 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:13.999724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:13.999820 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:13.999937 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:14.000185 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.000690 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.001536 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.001773 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.001883 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:14.002299 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:14.002725 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:14.002876 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:14.002948 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 16:07:14.003005 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:07:14.008978 20548 raft_consensus.cc:1261] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Refusing update from remote peer ed31894d5d5a4e8baf091ecbb6729e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:14.008992 20410 raft_consensus.cc:1261] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Refusing update from remote peer ed31894d5d5a4e8baf091ecbb6729e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:14.009718 20776 consensus_queue.cc:1035] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:14.009958 20773 consensus_queue.cc:1035] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 16:07:14.010893 20597 tablet.cc:2369] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:14.017858 20781 mvcc.cc:205] Tried to move back new op lower bound from 7332929675293540352 to 7332929675065466880. Current Snapshot: MvccSnapshot[applied={T|T < 7332929675293540352}]
I20260924 16:07:14.021498 20779 mvcc.cc:205] Tried to move back new op lower bound from 7332929675293540352 to 7332929675065466880. Current Snapshot: MvccSnapshot[applied={T|T < 7332929675293540352}]
I20260924 16:07:14.030485 19488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:14.030650 19488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:14.034751 19488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb4abb37975e499395cd083ddcfcd578: Sending DeleteTablet for 3 replicas of tablet fa21c108a1d04466a317b9eb52eae382
I20260924 16:07:14.035446 20667 tablet_service.cc:1514] Processing DeleteTablet for tablet fa21c108a1d04466a317b9eb52eae382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:14 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260924 16:07:14.035498 20527 tablet_service.cc:1514] Processing DeleteTablet for tablet fa21c108a1d04466a317b9eb52eae382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:14 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:47676
I20260924 16:07:14.035768 20389 tablet_service.cc:1514] Processing DeleteTablet for tablet fa21c108a1d04466a317b9eb52eae382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:14 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:34824
I20260924 16:07:14.035933 20789 tablet_replica.cc:330] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: stopping tablet replica
I20260924 16:07:14.036202 20790 tablet_replica.cc:330] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: stopping tablet replica
I20260924 16:07:14.036314 20789 raft_consensus.cc:2229] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:14.036626 20790 raft_consensus.cc:2229] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:14.036696 20789 raft_consensus.cc:2258] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:14.037060 20790 raft_consensus.cc:2258] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:14.038107 20789 ts_tablet_manager.cc:1905] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:14.038609 20790 ts_tablet_manager.cc:1905] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:14.040433 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19456
I20260924 16:07:14.041759 20789 ts_tablet_manager.cc:1918] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:14.041862 20789 log.cc:1196] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/fa21c108a1d04466a317b9eb52eae382
I20260924 16:07:14.042236 20789 ts_tablet_manager.cc:1939] T fa21c108a1d04466a317b9eb52eae382 P ed31894d5d5a4e8baf091ecbb6729e38: Deleting consensus metadata
I20260924 16:07:14.042253 20790 ts_tablet_manager.cc:1918] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:14.042460 20790 log.cc:1196] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/fa21c108a1d04466a317b9eb52eae382
I20260924 16:07:14.042800 20790 ts_tablet_manager.cc:1939] T fa21c108a1d04466a317b9eb52eae382 P e85af61ce1e240c9aa23fe66d1f328aa: Deleting consensus metadata
I20260924 16:07:14.046408 20788 tablet_replica.cc:330] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: stopping tablet replica
I20260924 16:07:14.046694 20788 raft_consensus.cc:2229] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:14.046989 20788 raft_consensus.cc:2258] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:14.048552 20788 ts_tablet_manager.cc:1905] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 16:07:14.048800 20344 connection.cc:535] server connection from 127.0.0.1:34824 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 16:07:14.049108 20623 connection.cc:535] server connection from 127.0.0.1:41904 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 16:07:14.052066 20788 ts_tablet_manager.cc:1918] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:14.052237 20788 log.cc:1196] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/fa21c108a1d04466a317b9eb52eae382
I20260924 16:07:14.052654 20788 ts_tablet_manager.cc:1939] T fa21c108a1d04466a317b9eb52eae382 P 254277422fba4b1e99b1fd7c7d20b47b: Deleting consensus metadata
W20260924 16:07:14.053882 20481 connection.cc:441] server connection from 127.0.0.1:47676 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47676 (request call id 1) could send its response
I20260924 16:07:14.058074 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:14.039335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.13.8.125:40327)
0924 16:07:14.039441 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:14.039445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:14.047483 (+ 8038us) server_negotiation.cc:407] Connection header received
0924 16:07:14.047692 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:14.047695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:14.047777 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:14.047833 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:14.049123 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.049873 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.051074 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.051413 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.051491 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:14.054012 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:14.054016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:14.054019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:14.054046 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:14.055864 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.056722 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.056727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.056729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.056816 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.057120 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.057123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.057124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.057330 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:14.057430 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:14.057798 (+ 368us) server_negotiation.cc:299] Negotiation successful
0924 16:07:14.057918 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 16:07:14.064810 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:14.053513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.13.8.125:40327)
0924 16:07:14.053629 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:14.053633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:14.054940 (+ 1307us) server_negotiation.cc:407] Connection header received
0924 16:07:14.054981 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:14.054984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:14.055026 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:14.055096 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:14.055924 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.056647 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.057959 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.058353 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.058416 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:14.061048 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:14.061054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:14.061060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:14.061088 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:14.062680 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.063464 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.063468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.063470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.063553 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.063876 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.063880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.063884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.064082 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:14.064180 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:14.064531 (+ 351us) server_negotiation.cc:299] Negotiation successful
0924 16:07:14.064645 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:07:14.065539 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:14.052559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.13.8.124:38309)
0924 16:07:14.052680 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:14.052684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:14.054190 (+ 1506us) server_negotiation.cc:407] Connection header received
0924 16:07:14.054372 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:14.054377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:14.054434 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:14.054489 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:14.055471 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.056256 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.057959 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.058326 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.058405 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:14.060424 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:14.060429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:14.060434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:14.060464 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:14.063738 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.064177 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.064180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.064182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.064280 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.064564 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.064567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.064568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.064757 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:14.064858 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:14.065259 (+ 401us) server_negotiation.cc:299] Negotiation successful
0924 16:07:14.065400 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:07:14.067615 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:14.039110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.13.8.124:38309)
0924 16:07:14.039226 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:14.039231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:14.058943 (+ 19712us) server_negotiation.cc:407] Connection header received
0924 16:07:14.059102 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:14.059105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:14.059152 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:14.059202 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:14.060052 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.060863 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.061954 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:14.062376 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:14.062489 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:14.063845 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:14.063849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:14.063852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:14.063879 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:14.065850 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.066414 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.066418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.066422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.066511 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:14.066768 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:14.066773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:14.066775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:14.066966 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:14.067084 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:14.067315 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 16:07:14.067418 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 16:07:15.021915 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.021392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:57431)
0924 16:07:15.021688 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.021793 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
W20260924 16:07:15.022459 20456 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:39979 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.022650 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.022179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:41677)
0924 16:07:15.022432 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.022521 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":114,"threads_started":1}
W20260924 16:07:15.022935 20734 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:39979 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.030395 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.029981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:50315)
0924 16:07:15.030197 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.030318 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":59,"threads_started":1}
W20260924 16:07:15.030685 20594 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:39979 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.535347 20802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb4abb37975e499395cd083ddcfcd578)
I20260924 16:07:15.535548 20802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } }
I20260924 16:07:15.536566 20802 leader_election.cc:288] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979), d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124:38309)
I20260924 16:07:15.536639 20803 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb4abb37975e499395cd083ddcfcd578)
I20260924 16:07:15.536896 20803 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } }
I20260924 16:07:15.537277 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.536682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51934)
0924 16:07:15.537039 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.537144 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":65,"threads_started":1}
W20260924 16:07:15.537513 20074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.538122 20803 leader_election.cc:288] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979), 6e60b5d591c7447fb748dd55655c0a12 (127.13.8.125:40327)
I20260924 16:07:15.538923 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.538191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51948)
0924 16:07:15.538750 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.538825 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":72,"threads_started":1}
I20260924 16:07:15.538936 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.538683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51950)
0924 16:07:15.538825 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.538872 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260924 16:07:15.539080 20074 leader_election.cc:334] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979): Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
W20260924 16:07:15.539122 20261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.539963 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.539752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51964)
0924 16:07:15.539857 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.539904 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260924 16:07:15.540069 20261 leader_election.cc:334] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979): Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.541154 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.536758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38309 (local address 127.0.0.1:37314)
0924 16:07:15.537118 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.537133 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.537255 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.537561 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.537565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.537584 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.537709 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.537715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.538748 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.538752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.540041 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.540054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.540719 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.540723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.540825 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.540847 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.540962 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.541034 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":64,"threads_started":1}
I20260924 16:07:15.541165 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.536916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.13.8.124:38309)
0924 16:07:15.537123 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.537128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.537212 (+ 84us) server_negotiation.cc:407] Connection header received
0924 16:07:15.537369 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.537372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.537431 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.537540 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.537819 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.538611 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.540151 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.540599 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.540717 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.540775 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.540977 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.541064 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
I20260924 16:07:15.541646 20278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e60b5d591c7447fb748dd55655c0a12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" is_pre_election: true
I20260924 16:07:15.541836 20278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e60b5d591c7447fb748dd55655c0a12 in term 1.
I20260924 16:07:15.542200 20076 leader_election.cc:302] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e60b5d591c7447fb748dd55655c0a12, d6a8a0943bd34f38b8beb0a07827b55b; no voters: bb4abb37975e499395cd083ddcfcd578
I20260924 16:07:15.542441 20802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 16:07:15.542498 20802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb4abb37975e499395cd083ddcfcd578)
I20260924 16:07:15.542539 20802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:07:15.543013 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.538493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:40327 (local address 127.0.0.1:37022)
0924 16:07:15.539316 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.539329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.539434 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.539754 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.539757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.539772 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.539846 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.539851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.540790 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.540796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.541824 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.541834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.542679 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.542683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.542774 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.542795 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.542867 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.542918 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":74,"threads_started":1}
I20260924 16:07:15.543136 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.538610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.13.8.125:40327)
0924 16:07:15.538883 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.538892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.539507 (+ 615us) server_negotiation.cc:407] Connection header received
0924 16:07:15.539545 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.539548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.539586 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.539653 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.539937 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.540675 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.541937 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.542445 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.542567 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.542622 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.542911 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.542982 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260924 16:07:15.543521 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e60b5d591c7447fb748dd55655c0a12" is_pre_election: true
I20260924 16:07:15.543898 20802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } }
I20260924 16:07:15.544142 20099 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d6a8a0943bd34f38b8beb0a07827b55b in current term 2: Already voted for candidate 6e60b5d591c7447fb748dd55655c0a12 in this term.
I20260924 16:07:15.544291 20802 leader_election.cc:288] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 election: Requested vote from peers bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979), d6a8a0943bd34f38b8beb0a07827b55b (127.13.8.124:38309)
I20260924 16:07:15.544562 20261 leader_election.cc:302] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a8a0943bd34f38b8beb0a07827b55b; no voters: 6e60b5d591c7447fb748dd55655c0a12, bb4abb37975e499395cd083ddcfcd578
I20260924 16:07:15.544601 20278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e60b5d591c7447fb748dd55655c0a12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6a8a0943bd34f38b8beb0a07827b55b"
I20260924 16:07:15.544690 20278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:07:15.544797 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.544513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51976)
0924 16:07:15.544685 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.544734 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260924 16:07:15.545464 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.545238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:51992)
0924 16:07:15.545347 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.545409 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260924 16:07:15.545616 20074 leader_election.cc:334] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979): Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
I20260924 16:07:15.545782 20278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e60b5d591c7447fb748dd55655c0a12 in term 2.
I20260924 16:07:15.545882 20803 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:07:15.546097 20076 leader_election.cc:302] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e60b5d591c7447fb748dd55655c0a12, d6a8a0943bd34f38b8beb0a07827b55b; no voters: bb4abb37975e499395cd083ddcfcd578
I20260924 16:07:15.546303 20802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 16:07:15.546469 20802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [term 2 LEADER]: Becoming Leader. State: Replica: 6e60b5d591c7447fb748dd55655c0a12, State: Running, Role: LEADER
I20260924 16:07:15.546702 20802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } }
I20260924 16:07:15.547658 20812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e60b5d591c7447fb748dd55655c0a12. Latest consensus state: current_term: 2 leader_uuid: "6e60b5d591c7447fb748dd55655c0a12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:15.547775 20812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:15.548058 20814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:15.550290 20814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:15.550693 20814 catalog_manager.cc:1259] Loaded cluster ID: ff812a99ca1b471c8013a8d455d27070
I20260924 16:07:15.550755 20814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:15.551403 20814 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:15.551797 20814 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Loaded TSK: 0
I20260924 16:07:15.552441 20814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:15.589464 20089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:15.592612 20089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:15.595813 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.595497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:52016)
0924 16:07:15.595609 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.595670 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 16:07:15.595912 20278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [term 2 FOLLOWER]: Refusing update from remote peer 6e60b5d591c7447fb748dd55655c0a12: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260924 16:07:15.596493 20812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 16:07:15.596954 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.596658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.0.0.1:52018)
0924 16:07:15.596830 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.596885 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260924 16:07:15.597076 20074 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 -> Peer bb4abb37975e499395cd083ddcfcd578 (127.13.8.126:39979): Couldn't send request to peer bb4abb37975e499395cd083ddcfcd578. Status: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 16:07:15.598173 20803 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e60b5d591c7447fb748dd55655c0a12. Latest consensus state: current_term: 2 leader_uuid: "6e60b5d591c7447fb748dd55655c0a12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:15.598342 20803 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:15.599468 20812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6e60b5d591c7447fb748dd55655c0a12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:15.599573 20812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:15.600113 20803 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6e60b5d591c7447fb748dd55655c0a12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bb4abb37975e499395cd083ddcfcd578" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 39979 } } peers { permanent_uuid: "6e60b5d591c7447fb748dd55655c0a12" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 40327 } attrs { promote: false } } peers { permanent_uuid: "d6a8a0943bd34f38b8beb0a07827b55b" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38309 } attrs { promote: false } } }
I20260924 16:07:15.600250 20803 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a8a0943bd34f38b8beb0a07827b55b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:07:15.609508 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.604993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41097 (local address 127.0.0.1:34826)
0924 16:07:15.605314 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.605330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.605419 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.605961 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.605964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.605981 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.606094 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.606099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.607037 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.607040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.608172 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.608181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.609041 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.609045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.609231 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.609257 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.609326 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.609389 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":48}
I20260924 16:07:15.609762 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.605577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.13.8.66:39245)
0924 16:07:15.605829 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.605833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.606177 (+ 344us) server_negotiation.cc:407] Connection header received
0924 16:07:15.606293 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.606296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.606346 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.606427 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.606753 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.607490 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.608587 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.609203 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.609329 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.609384 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.609543 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.609619 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260924 16:07:15.609941 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.605505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39245 (local address 127.0.0.1:47682)
0924 16:07:15.605979 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.605992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.606087 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.606542 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.606544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.606560 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.606637 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.606641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.607611 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.607614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.608454 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.608464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.609327 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.609330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.609413 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.609434 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.609527 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.609819 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"spinlock_wait_cycles":10112}
I20260924 16:07:15.610491 20527 tablet_service.cc:1467] Processing CreateTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e (DEFAULT_TABLE table=test-table [id=8ab8f90426ff452cab9ead77cab8c488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:15.610687 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.605114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.13.8.65:41097)
0924 16:07:15.605708 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.605714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.605728 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:07:15.605781 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.605786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.605849 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.605941 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.606212 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.606904 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.608302 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.608918 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.609025 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.609100 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.610434 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.610550 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":74,"threads_started":1}
I20260924 16:07:15.610795 20527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8bc83180c2b43ca90ac09fac2b30b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:15.610936 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.605749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45627 (local address 127.0.0.1:41918)
0924 16:07:15.605993 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.606007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.606078 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.607383 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.607386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.607404 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.607491 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.607497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.608507 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.608512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.609477 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.609486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.610565 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.610568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.610657 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.610678 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.610731 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.610817 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":45,"thread_start_us":91,"threads_started":1}
I20260924 16:07:15.611213 20389 tablet_service.cc:1467] Processing CreateTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e (DEFAULT_TABLE table=test-table [id=8ab8f90426ff452cab9ead77cab8c488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:15.611454 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8bc83180c2b43ca90ac09fac2b30b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:15.611526 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.606355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.13.8.67:45627)
0924 16:07:15.607156 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.607160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.607172 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:15.607219 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.607221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.607265 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.607604 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.607708 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.608404 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.609605 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.610441 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.611195 (+ 754us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.611257 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.611298 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.611393 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":94,"threads_started":1}
I20260924 16:07:15.613339 20825 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Bootstrap starting.
I20260924 16:07:15.613659 20667 tablet_service.cc:1467] Processing CreateTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e (DEFAULT_TABLE table=test-table [id=8ab8f90426ff452cab9ead77cab8c488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:15.613922 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8bc83180c2b43ca90ac09fac2b30b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:15.614138 20826 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: Bootstrap starting.
I20260924 16:07:15.614382 20825 tablet_bootstrap.cc:654] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:15.614986 20826 tablet_bootstrap.cc:654] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:15.615424 20825 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: No bootstrap required, opened a new log
I20260924 16:07:15.615531 20825 ts_tablet_manager.cc:1397] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 16:07:15.615937 20825 raft_consensus.cc:348] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.616060 20825 raft_consensus.cc:374] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:15.616124 20825 raft_consensus.cc:729] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254277422fba4b1e99b1fd7c7d20b47b, State: Initialized, Role: FOLLOWER
I20260924 16:07:15.616191 20826 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: No bootstrap required, opened a new log
I20260924 16:07:15.616297 20826 ts_tablet_manager.cc:1397] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:15.616329 20825 consensus_queue.cc:260] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.616683 20825 ts_tablet_manager.cc:1428] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 16:07:15.616791 20826 raft_consensus.cc:348] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.616925 20829 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Bootstrap starting.
I20260924 16:07:15.616952 20826 raft_consensus.cc:374] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:15.616989 20826 raft_consensus.cc:729] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85af61ce1e240c9aa23fe66d1f328aa, State: Initialized, Role: FOLLOWER
I20260924 16:07:15.617540 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.616989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:38159)
0924 16:07:15.617392 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.617447 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":83,"threads_started":1}
I20260924 16:07:15.617800 20829 tablet_bootstrap.cc:654] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:15.617125 20826 consensus_queue.cc:260] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.618166 20826 ts_tablet_manager.cc:1428] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:15.619024 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.618537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:38533)
0924 16:07:15.618872 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.618931 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":83,"threads_started":1}
I20260924 16:07:15.619943 20829 tablet_bootstrap.cc:492] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: No bootstrap required, opened a new log
I20260924 16:07:15.620031 20829 ts_tablet_manager.cc:1397] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 16:07:15.620205 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.619808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:50481)
0924 16:07:15.620075 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.620141 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260924 16:07:15.620476 20458 heartbeater.cc:503] Master 127.13.8.125:40327 was elected leader, sending a full tablet report...
I20260924 16:07:15.620488 20829 raft_consensus.cc:348] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.620592 20829 raft_consensus.cc:374] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:15.620630 20829 raft_consensus.cc:729] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed31894d5d5a4e8baf091ecbb6729e38, State: Initialized, Role: FOLLOWER
I20260924 16:07:15.620749 20829 consensus_queue.cc:260] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.621083 20829 ts_tablet_manager.cc:1428] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:15.621819 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.620317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:32953)
0924 16:07:15.621712 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.621759 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1330}
I20260924 16:07:15.622974 20736 heartbeater.cc:503] Master 127.13.8.125:40327 was elected leader, sending a full tablet report...
I20260924 16:07:15.622982 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.621800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:37413)
0924 16:07:15.622837 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.622895 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":82,"threads_started":1}
I20260924 16:07:15.830839 20832 raft_consensus.cc:482] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:15.830986 20832 raft_consensus.cc:504] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.832183 20832 leader_election.cc:288] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245), ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:15.835852 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.832420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39245 (local address 127.13.8.65:57889)
0924 16:07:15.832571 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.832618 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.832735 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.832978 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.832981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.832997 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.833113 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.833119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.833861 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.833867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.834757 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.834766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.835432 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.835436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.835543 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.835561 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.835649 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.835719 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260924 16:07:15.835852 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.832570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:57889 (local address 127.13.8.66:39245)
0924 16:07:15.832735 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.832743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.832763 (+ 20us) server_negotiation.cc:407] Connection header received
0924 16:07:15.832829 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.832834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.832890 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.832971 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.833228 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.833750 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.834884 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.835292 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.835411 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.835479 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.835665 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.835744 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 16:07:15.836063 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.832989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:55689 (local address 127.13.8.67:45627)
0924 16:07:15.833105 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.833109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.833125 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:07:15.833201 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.833209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.833249 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.833347 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:15.833620 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.834098 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.835092 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.835487 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.835608 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.835673 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.835844 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.835915 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:15.836270 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.832695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45627 (local address 127.13.8.65:55689)
0924 16:07:15.832996 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.833010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:15.833109 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:15.833366 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:15.833369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:15.833384 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:15.833493 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.833498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.834205 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.834210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.834964 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:15.834973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.835605 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.835609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:15.835720 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.835743 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:07:15.835828 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:07:15.835895 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":138,"threads_started":1}
I20260924 16:07:15.836282 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8bc83180c2b43ca90ac09fac2b30b3e" candidate_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "254277422fba4b1e99b1fd7c7d20b47b" is_pre_election: true
I20260924 16:07:15.836467 20548 raft_consensus.cc:2454] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e85af61ce1e240c9aa23fe66d1f328aa in term 0.
I20260924 16:07:15.836901 20343 leader_election.cc:302] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 254277422fba4b1e99b1fd7c7d20b47b, e85af61ce1e240c9aa23fe66d1f328aa; no voters:
I20260924 16:07:15.836918 20688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8bc83180c2b43ca90ac09fac2b30b3e" candidate_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" is_pre_election: true
I20260924 16:07:15.837131 20688 raft_consensus.cc:2454] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e85af61ce1e240c9aa23fe66d1f328aa in term 0.
I20260924 16:07:15.837189 20832 raft_consensus.cc:2781] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:15.837283 20832 raft_consensus.cc:482] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:15.837333 20832 raft_consensus.cc:3037] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:15.838452 20832 raft_consensus.cc:504] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.838822 20832 leader_election.cc:288] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [CANDIDATE]: Term 1 election: Requested vote from peers 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245), ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627)
I20260924 16:07:15.839236 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8bc83180c2b43ca90ac09fac2b30b3e" candidate_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "254277422fba4b1e99b1fd7c7d20b47b"
I20260924 16:07:15.839234 20688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8bc83180c2b43ca90ac09fac2b30b3e" candidate_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed31894d5d5a4e8baf091ecbb6729e38"
I20260924 16:07:15.839373 20548 raft_consensus.cc:3037] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:15.839396 20688 raft_consensus.cc:3037] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:15.840517 20688 raft_consensus.cc:2454] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e85af61ce1e240c9aa23fe66d1f328aa in term 1.
I20260924 16:07:15.840530 20548 raft_consensus.cc:2454] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e85af61ce1e240c9aa23fe66d1f328aa in term 1.
I20260924 16:07:15.840797 20343 leader_election.cc:302] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 254277422fba4b1e99b1fd7c7d20b47b, e85af61ce1e240c9aa23fe66d1f328aa; no voters:
I20260924 16:07:15.840942 20832 raft_consensus.cc:2781] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:15.841154 20832 raft_consensus.cc:686] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 1 LEADER]: Becoming Leader. State: Replica: e85af61ce1e240c9aa23fe66d1f328aa, State: Running, Role: LEADER
I20260924 16:07:15.841408 20832 consensus_queue.cc:237] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } }
I20260924 16:07:15.841778 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.841551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:38841)
0924 16:07:15.841664 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.841718 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 16:07:15.843616 20089 catalog_manager.cc:5495] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65). New cstate: current_term: 1 leader_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e85af61ce1e240c9aa23fe66d1f328aa" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 41097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:15.904894 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.901339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.13.8.65:41097)
0924 16:07:15.901481 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:15.901485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:15.901500 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:15.901624 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:15.901627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:15.901682 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:15.901793 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:15.902035 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.902630 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.903455 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:15.903690 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:15.903799 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:15.904056 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:15.904501 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:15.904635 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:15.904696 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 16:07:15.904750 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:15.907811 20548 raft_consensus.cc:1261] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Refusing update from remote peer e85af61ce1e240c9aa23fe66d1f328aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:15.908059 20688 raft_consensus.cc:1261] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Refusing update from remote peer e85af61ce1e240c9aa23fe66d1f328aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:15.908408 20838 consensus_queue.cc:1035] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "254277422fba4b1e99b1fd7c7d20b47b" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:15.908740 20832 consensus_queue.cc:1035] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed31894d5d5a4e8baf091ecbb6729e38" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:15.911128 20596 heartbeater.cc:503] Master 127.13.8.125:40327 was elected leader, sending a full tablet report...
I20260924 16:07:15.911641 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.910529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:58589)
0924 16:07:15.911477 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.911539 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":859}
I20260924 16:07:15.912139 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.911911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:54403)
0924 16:07:15.912015 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.912071 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 16:07:15.914409 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.913913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:56433)
0924 16:07:15.914293 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.914352 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
I20260924 16:07:15.915056 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.914820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.66:45227)
0924 16:07:15.914938 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.914991 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 16:07:15.915184 20842 mvcc.cc:205] Tried to move back new op lower bound from 7332929683073093632 to 7332929682809053184. Current Snapshot: MvccSnapshot[applied={T|T < 7332929683073093632 or (T in {7332929683073093632})}]
I20260924 16:07:15.917006 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.916326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.65:55267)
0924 16:07:15.916894 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.916945 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496}
I20260924 16:07:15.919602 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.919215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:39979 (local address 127.13.8.67:48903)
0924 16:07:15.919497 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.919542 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:39979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260924 16:07:15.922765 20089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:15.922905 20089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:15.926819 20089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e60b5d591c7447fb748dd55655c0a12: Sending DeleteTablet for 3 replicas of tablet f8bc83180c2b43ca90ac09fac2b30b3e
I20260924 16:07:15.928033 20527 tablet_service.cc:1514] Processing DeleteTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:15 UTC) from {username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:47682
I20260924 16:07:15.928072 20667 tablet_service.cc:1514] Processing DeleteTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:15 UTC) from {username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:41918
I20260924 16:07:15.928486 20852 tablet_replica.cc:330] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: stopping tablet replica
I20260924 16:07:15.928619 20852 raft_consensus.cc:2229] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:15.928610 20389 tablet_service.cc:1514] Processing DeleteTablet for tablet f8bc83180c2b43ca90ac09fac2b30b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:15 UTC) from {username='oryx', principal='oryx/127.13.8.125@KRBTEST.COM'} at 127.0.0.1:34826
I20260924 16:07:15.928761 20852 raft_consensus.cc:2258] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:15.929364 20854 tablet_replica.cc:330] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa: stopping tablet replica
I20260924 16:07:15.929477 20854 raft_consensus.cc:2229] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:15.929527 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20236
I20260924 16:07:15.929803 20854 raft_consensus.cc:2258] T f8bc83180c2b43ca90ac09fac2b30b3e P e85af61ce1e240c9aa23fe66d1f328aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:15.930042 20853 tablet_replica.cc:330] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: stopping tablet replica
I20260924 16:07:15.930181 20853 raft_consensus.cc:2229] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:15.930356 20853 raft_consensus.cc:2258] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:15.930773 20852 ts_tablet_manager.cc:1905] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:15.931725 20853 ts_tablet_manager.cc:1905] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:15.933462 20852 ts_tablet_manager.cc:1918] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:15.933547 20852 log.cc:1196] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/f8bc83180c2b43ca90ac09fac2b30b3e
I20260924 16:07:15.933808 20852 ts_tablet_manager.cc:1939] T f8bc83180c2b43ca90ac09fac2b30b3e P 254277422fba4b1e99b1fd7c7d20b47b: Deleting consensus metadata
I20260924 16:07:15.934536 20853 ts_tablet_manager.cc:1918] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:15.934612 20853 log.cc:1196] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/f8bc83180c2b43ca90ac09fac2b30b3e
I20260924 16:07:15.934651 20074 catalog_manager.cc:4841] TS 254277422fba4b1e99b1fd7c7d20b47b (127.13.8.66:39245): tablet f8bc83180c2b43ca90ac09fac2b30b3e (table test-table [id=8ab8f90426ff452cab9ead77cab8c488]) successfully deleted
I20260924 16:07:15.934815 20853 ts_tablet_manager.cc:1939] T f8bc83180c2b43ca90ac09fac2b30b3e P ed31894d5d5a4e8baf091ecbb6729e38: Deleting consensus metadata
I20260924 16:07:15.935384 20074 catalog_manager.cc:4841] TS ed31894d5d5a4e8baf091ecbb6729e38 (127.13.8.67:45627): tablet f8bc83180c2b43ca90ac09fac2b30b3e (table test-table [id=8ab8f90426ff452cab9ead77cab8c488]) successfully deleted
I20260924 16:07:15.937845 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20464
I20260924 16:07:15.938036 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:15.937686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:41097 (local address 127.0.0.1:34856)
0924 16:07:15.937832 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:15.937923 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260924 16:07:15.938274 20076 catalog_manager.cc:4584] TS e85af61ce1e240c9aa23fe66d1f328aa (127.13.8.65:41097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8bc83180c2b43ca90ac09fac2b30b3e: Network error: Client connection negotiation failed: client connection to 127.13.8.65:41097: connect: Connection refused (error 111)
I20260924 16:07:15.945330 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20601
I20260924 16:07:15.952884 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 19952
I20260924 16:07:15.958911 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20059
2026-09-24T16:07:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5631 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:16 dist-test-slave-vj6n krb5kdc[20857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:16 dist-test-slave-vj6n krb5kdc[20857](info): set up 2 sockets
Sep 24 16:07:16 dist-test-slave-vj6n krb5kdc[20857](info): commencing operation
krb5kdc: starting...
W20260924 16:07:17.004513 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab.
Entry for principal oryx/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab.
I20260924 16:07:17.126116 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:33359
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab
--principal=oryx/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:33359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:17.238689 20871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:17.238919 20871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:17.238957 20871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:17.242970 20871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:17.243014 20871 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 16:07:17.243031 20871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:17.243048 20871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:17.243063 20871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:17.243077 20871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:17.248175 20871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:33359
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.126.keytab
--principal=oryx/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:17.249104 20871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:17.250032 20871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:17.258255 20876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.258369 20871 server_base.cc:1034] running on GCE node
W20260924 16:07:17.258473 20877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:17.258250 20879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.259042 20871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:17.260167 20871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:17.261353 20871 hybrid_clock.cc:648] HybridClock initialized: now 1790266037261321 us; error 44 us; skew 500 ppm
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:17.264050 20871 init.cc:377] Logged in from keytab as oryx/127.13.8.126@KRBTEST.COM (short username oryx)
I20260924 16:07:17.264863 20871 webserver.cc:460] Webserver started at http://127.13.8.126:42739/ using document root <none> and password file <none>
I20260924 16:07:17.265403 20871 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:17.265460 20871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:17.265616 20871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:17.267567 20871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "c643edeee0ed498cb00fdcc47445b386"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "e93e42cb018b1fca65d2f58b945cf977"
server_key_iv: "bbc73b106f59c9ed0911f49a293acafc"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.268061 20871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "c643edeee0ed498cb00fdcc47445b386"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "e93e42cb018b1fca65d2f58b945cf977"
server_key_iv: "bbc73b106f59c9ed0911f49a293acafc"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.271696 20871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:07:17.274003 20886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.275202 20871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:17.275345 20871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "c643edeee0ed498cb00fdcc47445b386"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "e93e42cb018b1fca65d2f58b945cf977"
server_key_iv: "bbc73b106f59c9ed0911f49a293acafc"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.275449 20871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:17.287137 20871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:17.290486 20871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:17.290680 20871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:17.298439 20871 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:33359
I20260924 16:07:17.298517 20949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:33359 every 8 connection(s)
I20260924 16:07:17.299382 20871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:17.301851 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20871
I20260924 16:07:17.301973 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:17.302302 13345 external_mini_cluster.cc:1428] Setting key c31468e12ba135e04ff8dfa1be76d35d
I20260924 16:07:17.302327 20950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:17.308799 20950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Bootstrap starting.
I20260924 16:07:17.310984 20950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Neither blocks nor log segments found. Creating new log.
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:17.311678 20950 log.cc:824] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:17.313498 20950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: No bootstrap required, opened a new log
I20260924 16:07:17.316480 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.303689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.13.8.126:33359)
0924 16:07:17.304137 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.304152 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.304186 (+ 34us) server_negotiation.cc:407] Connection header received
0924 16:07:17.305121 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.305139 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.305459 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.305779 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.306579 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.307596 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.308449 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.308976 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.309150 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.311554 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:17.311559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:17.311574 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:17.311600 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:17.313754 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.314163 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.314171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.314180 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.314348 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.314627 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.314632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.314634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.314995 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:17.315196 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:17.315491 (+ 295us) server_negotiation.cc:299] Negotiation successful
0924 16:07:17.315761 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":151,"threads_started":1}
I20260924 16:07:17.316388 20950 raft_consensus.cc:348] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } }
I20260924 16:07:17.316630 20950 raft_consensus.cc:374] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:17.316691 20950 raft_consensus.cc:729] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c643edeee0ed498cb00fdcc47445b386, State: Initialized, Role: FOLLOWER
I20260924 16:07:17.317145 20950 consensus_queue.cc:260] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } }
I20260924 16:07:17.317265 20950 raft_consensus.cc:388] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:17.317348 20950 raft_consensus.cc:482] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:17.317446 20950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:17.318382 20950 raft_consensus.cc:504] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } }
I20260924 16:07:17.318745 20950 leader_election.cc:302] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c643edeee0ed498cb00fdcc47445b386; no voters:
I20260924 16:07:17.319077 20950 leader_election.cc:288] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:17.319161 20955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:17.319409 20955 raft_consensus.cc:686] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [term 1 LEADER]: Becoming Leader. State: Replica: c643edeee0ed498cb00fdcc47445b386, State: Running, Role: LEADER
I20260924 16:07:17.319779 20955 consensus_queue.cc:237] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } }
I20260924 16:07:17.320442 20950 sys_catalog.cc:564] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:17.321626 20957 sys_catalog.cc:455] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c643edeee0ed498cb00fdcc47445b386. Latest consensus state: current_term: 1 leader_uuid: "c643edeee0ed498cb00fdcc47445b386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } } }
I20260924 16:07:17.321637 20956 sys_catalog.cc:455] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c643edeee0ed498cb00fdcc47445b386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c643edeee0ed498cb00fdcc47445b386" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33359 } } }
I20260924 16:07:17.321756 20957 sys_catalog.cc:458] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:17.321755 20956 sys_catalog.cc:458] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:17.322108 20964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:17.325014 20964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:17.330829 20964 catalog_manager.cc:1347] Generated new cluster ID: d653a7ab58c64e2d80c9586685a948b9
I20260924 16:07:17.330888 20964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:17.341475 20964 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:17.342739 20964 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:17.354338 20964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Generated new TSK 0
I20260924 16:07:17.355170 20964 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
I20260924 16:07:17.385802 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:33359
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:17.496466 20976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:17.496901 20976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:17.496989 20976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:17.501194 20976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:17.501248 20976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:17.501287 20976 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:07:17.501353 20976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:17.506484 20976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:17.507496 20976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:17.508522 20976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:17.517895 20981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:17.517952 20982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.518030 20976 server_base.cc:1034] running on GCE node
W20260924 16:07:17.517946 20984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.518579 20976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:17.519215 20976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:17.520383 20976 hybrid_clock.cc:648] HybridClock initialized: now 1790266037520365 us; error 29 us; skew 500 ppm
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:17.523259 20976 init.cc:377] Logged in from keytab as oryx/127.13.8.65@KRBTEST.COM (short username oryx)
I20260924 16:07:17.524142 20976 webserver.cc:460] Webserver started at http://127.13.8.65:36339/ using document root <none> and password file <none>
I20260924 16:07:17.524746 20976 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:17.524803 20976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:17.525033 20976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:17.527132 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "5f0cd4a4430b40b48590d96cc59838c5"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6ea042c42e13d3323751afaff88f78e5"
server_key_iv: "92173b8964fa22e22b6acbf5cd9b6692"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.527706 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "5f0cd4a4430b40b48590d96cc59838c5"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6ea042c42e13d3323751afaff88f78e5"
server_key_iv: "92173b8964fa22e22b6acbf5cd9b6692"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.531303 20976 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:07:17.533706 20991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.534893 20976 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:17.535032 20976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "5f0cd4a4430b40b48590d96cc59838c5"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6ea042c42e13d3323751afaff88f78e5"
server_key_iv: "92173b8964fa22e22b6acbf5cd9b6692"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.535141 20976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:17.548625 20976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:17.551786 20976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:17.551975 20976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:17.553368 20976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:17.553433 20976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.553500 20976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:17.553560 20976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:17.564623 20976 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:40767
I20260924 16:07:17.564689 21107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:40767 every 8 connection(s)
I20260924 16:07:17.565555 20976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:17.567984 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.554065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36121 (local address 127.13.8.126:33359)
0924 16:07:17.554205 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.554211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.555147 (+ 936us) server_negotiation.cc:407] Connection header received
0924 16:07:17.556431 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.556438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.556505 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.556569 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.558419 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.559216 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.561060 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.561491 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.561587 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.564500 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:17.564504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:17.564508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:17.564538 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:17.566125 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.566740 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.566743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.566744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.566818 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.567112 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.567114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.567115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.567275 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:17.567346 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:17.567775 (+ 429us) server_negotiation.cc:299] Negotiation successful
0924 16:07:17.567870 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:17.568748 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.554496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.65:36121)
0924 16:07:17.554970 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:17.555025 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:17.556124 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:17.557013 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:17.557023 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:17.557510 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:17.558259 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.558284 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.559390 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.559393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.560917 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.560930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.561645 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.561651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.561858 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.562533 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:17.562557 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:17.564312 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:17.566306 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.566312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.566325 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.566640 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.566900 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.566902 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.566903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.567026 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.567359 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:17.567364 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:17.567571 (+ 207us) client_negotiation.cc:769] Sending connection context
0924 16:07:17.567712 (+ 141us) client_negotiation.cc:240] Negotiation successful
0924 16:07:17.567941 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260924 16:07:17.570073 21108 heartbeater.cc:347] Connected to a master server at 127.13.8.126:33359
I20260924 16:07:17.570274 21108 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:17.570777 21108 heartbeater.cc:511] Master 127.13.8.126:33359 requested a full tablet report, sending...
I20260924 16:07:17.571758 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 20976
I20260924 16:07:17.571884 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:17.572248 13345 external_mini_cluster.cc:1428] Setting key 448a68ee0439f9181d7b8585d2a552cf
I20260924 16:07:17.572495 20900 ts_manager.cc:194] Registered new tserver with Master: 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767)
I20260924 16:07:17.574069 20900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:36121
I20260924 16:07:17.579768 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.572401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.65:59591)
0924 16:07:17.572725 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:17.572740 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:17.572824 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:17.573052 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:17.573054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:17.573249 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:17.573364 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.573371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.574168 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.574190 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.574846 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.574855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.575424 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.575427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.575556 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.576044 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:17.576055 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:17.576384 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:17.578012 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.578018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.578023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.578354 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.578611 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.578616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.578618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.578731 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.579264 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:17.579267 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:17.579345 (+ 78us) client_negotiation.cc:769] Sending connection context
0924 16:07:17.579435 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 16:07:17.579533 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":19,"thread_start_us":93,"threads_started":1}
I20260924 16:07:17.579877 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.572707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59591 (local address 127.13.8.126:33359)
0924 16:07:17.572818 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.572822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.572834 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:17.572898 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.572901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.572944 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.573021 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.573522 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.574037 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.574959 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.575301 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.575420 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.576524 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:17.576527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:17.576530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:17.576552 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:17.577868 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.578446 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.578449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.578450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.578522 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.578890 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.578896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.578901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.579151 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:17.579266 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:17.579524 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 16:07:17.579654 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
I20260924 16:07:17.601266 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:33359
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:17.710986 21117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:17.711266 21117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:17.711311 21117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:17.715359 21117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:17.715415 21117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:17.715454 21117 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:07:17.715520 21117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:17.720916 21117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:17.722113 21117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:17.723181 21117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:17.733029 21125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.733134 21117 server_base.cc:1034] running on GCE node
W20260924 16:07:17.733044 21123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:17.733079 21122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.733904 21117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:17.734738 21117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:17.735917 21117 hybrid_clock.cc:648] HybridClock initialized: now 1790266037735862 us; error 52 us; skew 500 ppm
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:17.738905 21117 init.cc:377] Logged in from keytab as oryx/127.13.8.66@KRBTEST.COM (short username oryx)
I20260924 16:07:17.739807 21117 webserver.cc:460] Webserver started at http://127.13.8.66:38951/ using document root <none> and password file <none>
I20260924 16:07:17.740397 21117 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:17.740487 21117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:17.740700 21117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:17.742828 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "213f00b51f714189922951d6479a9366"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6afeed71d7f0f58544b0f7994ae339e7"
server_key_iv: "ecff42ea8faec3c2c663d35104359e46"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.743362 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "213f00b51f714189922951d6479a9366"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6afeed71d7f0f58544b0f7994ae339e7"
server_key_iv: "ecff42ea8faec3c2c663d35104359e46"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.747262 21117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:07:17.749773 21132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.751082 21117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:17.751220 21117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "213f00b51f714189922951d6479a9366"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "6afeed71d7f0f58544b0f7994ae339e7"
server_key_iv: "ecff42ea8faec3c2c663d35104359e46"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.751341 21117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:17.767277 21117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:17.770118 21117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:17.770370 21117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:17.771857 21117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:17.771930 21117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.772004 21117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:17.772050 21117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:17.783202 21117 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38351
I20260924 16:07:17.783294 21248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38351 every 8 connection(s)
I20260924 16:07:17.784415 21117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:17.786808 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.772845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:56731 (local address 127.13.8.126:33359)
0924 16:07:17.772971 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.772974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.774090 (+ 1116us) server_negotiation.cc:407] Connection header received
0924 16:07:17.775318 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.775322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.775393 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.775495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.777177 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.777777 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.779449 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.779733 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.779852 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.783031 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:17.783034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:17.783037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:17.783068 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:17.784710 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.785336 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.785339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.785340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.785414 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.785782 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.785785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.785786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.785966 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:17.786040 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:17.786589 (+ 549us) server_negotiation.cc:299] Negotiation successful
0924 16:07:17.786688 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:07:17.787447 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21117
I20260924 16:07:17.787597 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:17.787767 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.773417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.66:56731)
0924 16:07:17.773913 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:17.773957 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:17.775027 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:17.775742 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:17.775752 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:17.776198 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:17.776978 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.777000 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.777917 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.777922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.779294 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.779307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.779915 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.779919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.780149 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.780753 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:17.780777 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:17.782857 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:17.784858 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.784864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.784881 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.785228 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.785509 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.785512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.785514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.785669 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.786066 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:17.786071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:17.786369 (+ 298us) client_negotiation.cc:769] Sending connection context
0924 16:07:17.786520 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 16:07:17.786767 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":138,"threads_started":1}
I20260924 16:07:17.787928 13345 external_mini_cluster.cc:1428] Setting key 40d4c75bfddadfaf6e9addb360c913cd
I20260924 16:07:17.789260 21249 heartbeater.cc:347] Connected to a master server at 127.13.8.126:33359
I20260924 16:07:17.789417 21249 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:17.789954 21249 heartbeater.cc:511] Master 127.13.8.126:33359 requested a full tablet report, sending...
I20260924 16:07:17.793314 20903 ts_manager.cc:194] Registered new tserver with Master: 213f00b51f714189922951d6479a9366 (127.13.8.66:38351)
I20260924 16:07:17.793882 20903 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:56731
WARNING: no policy specified for oryx/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:17.800310 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.791397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:50607 (local address 127.13.8.126:33359)
0924 16:07:17.791514 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.791518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.791831 (+ 313us) server_negotiation.cc:407] Connection header received
0924 16:07:17.791945 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.791947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.791981 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.792073 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.792537 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.793050 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.795196 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.795640 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.795775 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.796920 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:17.796926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:17.796931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:17.796963 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:17.798624 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.799021 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.799028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.799031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.799154 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:17.799536 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:17.799539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:17.799540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:17.799769 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:17.799850 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:17.800103 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 16:07:17.800195 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:07:17.800382 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.791343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.66:50607)
0924 16:07:17.791743 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:17.791760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:17.791864 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:17.792090 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:17.792093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:17.792296 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:17.792385 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.792413 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.794258 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.794262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.795065 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.795076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.795781 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.795785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.795874 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.796454 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:17.796465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:17.796778 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:17.798731 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.798733 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.798735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.798921 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.799267 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:17.799272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:17.799275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:17.799377 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:17.799873 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:17.799875 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:17.799957 (+ 82us) client_negotiation.cc:769] Sending connection context
0924 16:07:17.800046 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 16:07:17.800166 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
I20260924 16:07:17.817296 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:33359
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:17.927532 21258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:17.927850 21258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:17.927925 21258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:17.931915 21258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:17.932008 21258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:17.932086 21258 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 16:07:17.932166 21258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:17.937507 21258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.8.126:33359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:17.938727 21258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:17.939880 21258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:17.949736 21266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:17.949719 21264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:17.949719 21263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:17.949951 21258 server_base.cc:1034] running on GCE node
I20260924 16:07:17.950488 21258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:17.951179 21258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:17.952378 21258 hybrid_clock.cc:648] HybridClock initialized: now 1790266037952350 us; error 50 us; skew 500 ppm
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:17.955196 21258 init.cc:377] Logged in from keytab as oryx/127.13.8.67@KRBTEST.COM (short username oryx)
I20260924 16:07:17.956189 21258 webserver.cc:460] Webserver started at http://127.13.8.67:39525/ using document root <none> and password file <none>
I20260924 16:07:17.956756 21258 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:17.956816 21258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:17.956987 21258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:17.959076 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "2b88cd2cb3864003a16cf3282551d770"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "a0a72779cfc1895d67693b79877033ab"
server_key_iv: "e98ebbddc76d6904d94731f73338059f"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.959604 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "2b88cd2cb3864003a16cf3282551d770"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "a0a72779cfc1895d67693b79877033ab"
server_key_iv: "e98ebbddc76d6904d94731f73338059f"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.963330 21258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:07:17.965812 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.967181 21258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:17.967319 21258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "2b88cd2cb3864003a16cf3282551d770"
format_stamp: "Formatted at 2026-09-24 16:07:17 on dist-test-slave-vj6n"
server_key: "a0a72779cfc1895d67693b79877033ab"
server_key_iv: "e98ebbddc76d6904d94731f73338059f"
server_key_version: "encryptionkey@0"
I20260924 16:07:17.967437 21258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:17.985378 21258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:17.988164 21258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:17.988364 21258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:17.989985 21258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:17.990064 21258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:17.990161 21258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:17.990237 21258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 24 16:07:17 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:18.001001 21258 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:33295
I20260924 16:07:18.001046 21389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:33295 every 8 connection(s)
I20260924 16:07:18.002002 21258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:18.003014 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21258
I20260924 16:07:18.003140 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:18.003499 13345 external_mini_cluster.cc:1428] Setting key 8a8d0d53e5eba3774d431153ad5a1981
I20260924 16:07:18.007115 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.990755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:52483 (local address 127.13.8.126:33359)
0924 16:07:17.990878 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:17.990884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:17.991961 (+ 1077us) server_negotiation.cc:407] Connection header received
0924 16:07:17.993038 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:17.993043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:17.993108 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:17.993170 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:17.995094 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.996022 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.997902 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.998172 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.998338 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.001388 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:18.001392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:18.001395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:18.001428 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:18.004485 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.005137 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.005144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.005149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.005299 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.005824 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.005830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.005835 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.006188 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:18.006331 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.006891 (+ 560us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.006997 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:07:18.008006 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:17.991331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.67:52483)
0924 16:07:17.991816 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:17.991859 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:17.992766 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:17.993665 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:17.993690 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:17.994266 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:17.994919 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.994939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.996179 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.996183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.997775 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:17.997788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:17.998381 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:17.998385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:17.998569 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:17.999195 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:17.999224 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:18.001217 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:18.004610 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.004617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.004637 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.005011 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.005419 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.005423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.005425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.005708 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.006390 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:18.006397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:18.006656 (+ 259us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.006823 (+ 167us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.007093 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":151,"threads_started":1}
I20260924 16:07:18.009373 21390 heartbeater.cc:347] Connected to a master server at 127.13.8.126:33359
I20260924 16:07:18.009544 21390 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:18.010090 21390 heartbeater.cc:511] Master 127.13.8.126:33359 requested a full tablet report, sending...
I20260924 16:07:18.011140 20900 ts_manager.cc:194] Registered new tserver with Master: 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:18.011916 20900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:52483
I20260924 16:07:18.018162 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:07:18.019121 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.011547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:49649 (local address 127.13.8.126:33359)
0924 16:07:18.011679 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.011684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.011858 (+ 174us) server_negotiation.cc:407] Connection header received
0924 16:07:18.011983 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.011985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.012039 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.012115 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.012590 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.013077 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.014173 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.014434 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.014511 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.015542 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:18.015545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:18.015546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:18.015568 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:18.017386 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.017873 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.017876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.017878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.017969 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.018318 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.018321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.018323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.018496 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:18.018611 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.018862 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.018969 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:18.019142 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.011468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.13.8.67:49649)
0924 16:07:18.011789 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.011804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.011902 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.012163 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.012166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.012365 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:18.012490 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.012498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.013221 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.013224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.014066 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.014075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.014526 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.014529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.014603 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.015077 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:18.015090 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:18.015417 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:18.017501 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.017505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.017507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.017754 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.018070 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.018074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.018076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.018178 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.018606 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:18.018609 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:18.018706 (+ 97us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.018810 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.018958 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":140,"threads_started":1}
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:18.043249 20953 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:18.036496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.13.8.126:33359)
0924 16:07:18.036650 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.036654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.036670 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:07:18.036721 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.036724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.036771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.036877 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.037614 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.038303 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.039243 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.039488 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.039717 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.043044 (+ 3327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58434: BlockingRecv error: recv got EOF from 127.0.0.1:58434 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:18.052062 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.044693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.13.8.126:33359)
0924 16:07:18.044829 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.044833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.044846 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:18.044953 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.044955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.044992 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.045055 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.045602 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.046073 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.046914 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.047111 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.047186 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.049377 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:18.049380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:18.049382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:18.049401 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:18.050644 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.050999 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.051001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.051002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.051071 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.051293 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.051295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.051296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.051455 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:18.051554 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.051768 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.051851 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:07:18.054870 20900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:18.057842 20900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:18.071815 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.066051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40767 (local address 127.0.0.1:57482)
0924 16:07:18.066459 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.066498 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.066595 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.067224 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.067230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.067255 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.067393 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.067400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.068803 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.068806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.070152 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.070165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.071278 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.071285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.071396 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.071442 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.071537 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.071666 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
I20260924 16:07:18.072620 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.066266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57482 (local address 127.13.8.65:40767)
0924 16:07:18.066636 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.066651 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.066676 (+ 25us) server_negotiation.cc:407] Connection header received
0924 16:07:18.066757 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.066765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.067026 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.067208 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.067529 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.068688 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.070302 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.071130 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.072141 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.072298 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.072350 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.072441 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":83,"threads_started":1}
I20260924 16:07:18.073014 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.066878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38351 (local address 127.0.0.1:51338)
0924 16:07:18.067603 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.067613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.067739 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.068336 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.068339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.068355 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.068448 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.068457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.069874 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.069877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.071212 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.071223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.072400 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.072404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.072490 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.072518 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.072597 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.072663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":112,"threads_started":1}
I20260924 16:07:18.073153 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.067637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.13.8.67:33295)
0924 16:07:18.068189 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.068200 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.068505 (+ 305us) server_negotiation.cc:407] Connection header received
0924 16:07:18.068575 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.068579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.068832 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.068988 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.069204 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.070056 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.071492 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.072260 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.072371 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.072523 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.072908 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.073000 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":120,"threads_started":1}
I20260924 16:07:18.073041 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.067229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.0.0.1:60982)
0924 16:07:18.068321 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.068334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.068422 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.068967 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.068970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.068988 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.069078 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.069085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.070199 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.070204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.071361 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.071371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.072584 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.072590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.072669 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.072692 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.072775 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.072829 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260924 16:07:18.075873 21042 tablet_service.cc:1467] Processing CreateTablet for tablet c085847ffcec4763998aec3fadce8400 (DEFAULT_TABLE table=test-table [id=cd153d82452b44ea8a980556e3dc0f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.075875 21324 tablet_service.cc:1467] Processing CreateTablet for tablet c085847ffcec4763998aec3fadce8400 (DEFAULT_TABLE table=test-table [id=cd153d82452b44ea8a980556e3dc0f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.076823 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c085847ffcec4763998aec3fadce8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.076823 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c085847ffcec4763998aec3fadce8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.082485 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.067082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.13.8.66:38351)
0924 16:07:18.067507 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.067521 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.067734 (+ 213us) server_negotiation.cc:407] Connection header received
0924 16:07:18.067845 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.067850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.068036 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.068195 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.068574 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.069753 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.071342 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.072232 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.072341 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.072480 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.072642 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.082330 (+ 9688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":100,"threads_started":1}
I20260924 16:07:18.083058 21418 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Bootstrap starting.
I20260924 16:07:18.085239 21418 tablet_bootstrap.cc:654] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.085345 21183 tablet_service.cc:1467] Processing CreateTablet for tablet c085847ffcec4763998aec3fadce8400 (DEFAULT_TABLE table=test-table [id=cd153d82452b44ea8a980556e3dc0f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.086179 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c085847ffcec4763998aec3fadce8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.086211 21418 log.cc:824] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:18.086872 21419 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Bootstrap starting.
I20260924 16:07:18.088282 21418 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: No bootstrap required, opened a new log
I20260924 16:07:18.088472 21418 ts_tablet_manager.cc:1397] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:07:18.089092 21419 tablet_bootstrap.cc:654] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.089922 21419 log.cc:824] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:18.091956 21419 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: No bootstrap required, opened a new log
I20260924 16:07:18.092159 21419 ts_tablet_manager.cc:1397] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260924 16:07:18.092381 21418 raft_consensus.cc:348] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.092634 21418 raft_consensus.cc:374] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.092684 21418 raft_consensus.cc:729] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.093124 21418 consensus_queue.cc:260] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.093456 21422 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Bootstrap starting.
I20260924 16:07:18.094103 21418 ts_tablet_manager.cc:1428] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:07:18.094117 21108 heartbeater.cc:503] Master 127.13.8.126:33359 was elected leader, sending a full tablet report...
I20260924 16:07:18.095644 21422 tablet_bootstrap.cc:654] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.095762 21419 raft_consensus.cc:348] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.096009 21419 raft_consensus.cc:374] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.096069 21419 raft_consensus.cc:729] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.096508 21422 log.cc:824] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:18.096531 21419 consensus_queue.cc:260] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.097246 21390 heartbeater.cc:503] Master 127.13.8.126:33359 was elected leader, sending a full tablet report...
I20260924 16:07:18.097539 21419 ts_tablet_manager.cc:1428] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260924 16:07:18.099288 21422 tablet_bootstrap.cc:492] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: No bootstrap required, opened a new log
I20260924 16:07:18.099491 21422 ts_tablet_manager.cc:1397] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:07:18.102504 21422 raft_consensus.cc:348] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.102730 21422 raft_consensus.cc:374] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.102849 21422 raft_consensus.cc:729] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213f00b51f714189922951d6479a9366, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.103237 21422 consensus_queue.cc:260] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.103839 21249 heartbeater.cc:503] Master 127.13.8.126:33359 was elected leader, sending a full tablet report...
I20260924 16:07:18.104147 21422 ts_tablet_manager.cc:1428] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260924 16:07:18.115736 21425 raft_consensus.cc:482] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:18.115904 21425 raft_consensus.cc:504] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.116919 21425 leader_election.cc:288] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 213f00b51f714189922951d6479a9366 (127.13.8.66:38351)
I20260924 16:07:18.120970 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.117251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:56665 (local address 127.13.8.65:40767)
0924 16:07:18.117410 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.117415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.117431 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:07:18.117554 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.117558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.117636 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.117715 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.118071 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.118815 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.119849 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.120373 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.120489 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.120552 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.120762 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.120858 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:07:18.120972 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.117160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40767 (local address 127.13.8.67:56665)
0924 16:07:18.117303 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.117321 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.117433 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.117798 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.117802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.117823 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.117956 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.117964 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.118927 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.118934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.119724 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.119734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.120490 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.120493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.120637 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.120655 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.120745 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.120839 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":26}
I20260924 16:07:18.121672 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c085847ffcec4763998aec3fadce8400" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5" is_pre_election: true
I20260924 16:07:18.121814 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.117596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:42393 (local address 127.13.8.66:38351)
0924 16:07:18.117721 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.117725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.117947 (+ 222us) server_negotiation.cc:407] Connection header received
0924 16:07:18.118027 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.118030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.118085 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.118164 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.118766 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.119252 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.120728 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.121270 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.121403 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.121458 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.121641 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.121713 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:07:18.121827 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.117458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38351 (local address 127.13.8.67:42393)
0924 16:07:18.117837 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.117850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.117944 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.118178 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.118181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.118198 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.118599 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.118607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.119364 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.119369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.120601 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.120611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.121416 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.121420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.121522 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.121538 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.121627 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.121689 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":19,"thread_start_us":117,"threads_started":1}
I20260924 16:07:18.122016 21062 raft_consensus.cc:2454] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 0.
I20260924 16:07:18.122519 21274 leader_election.cc:302] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b88cd2cb3864003a16cf3282551d770, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:18.122804 21425 raft_consensus.cc:2781] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:18.122802 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c085847ffcec4763998aec3fadce8400" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366" is_pre_election: true
I20260924 16:07:18.122903 21425 raft_consensus.cc:482] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:18.122951 21425 raft_consensus.cc:3037] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.123073 21203 raft_consensus.cc:2454] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 0.
I20260924 16:07:18.124035 21425 raft_consensus.cc:504] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.124327 21425 leader_election.cc:288] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 election: Requested vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 213f00b51f714189922951d6479a9366 (127.13.8.66:38351)
I20260924 16:07:18.124716 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c085847ffcec4763998aec3fadce8400" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366"
I20260924 16:07:18.124720 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c085847ffcec4763998aec3fadce8400" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5"
I20260924 16:07:18.124850 21203 raft_consensus.cc:3037] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.124877 21062 raft_consensus.cc:3037] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.125866 21062 raft_consensus.cc:2454] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 1.
I20260924 16:07:18.126168 21274 leader_election.cc:302] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b88cd2cb3864003a16cf3282551d770, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:18.126358 21425 raft_consensus.cc:2781] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:18.126394 21203 raft_consensus.cc:2454] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 1.
I20260924 16:07:18.126616 21425 raft_consensus.cc:686] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 1 LEADER]: Becoming Leader. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Running, Role: LEADER
I20260924 16:07:18.126957 21425 consensus_queue.cc:237] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.129941 20903 catalog_manager.cc:5495] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } health_report { overall_health: HEALTHY } } }
I20260924 16:07:18.154990 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.151260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.13.8.67:33295)
0924 16:07:18.151404 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.151409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.151424 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:18.151614 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.151617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.151674 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.151794 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:18.152040 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.152564 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.153436 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.153667 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.153774 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.154065 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.154580 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.154747 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.154781 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.154839 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:07:18.161156 21203 raft_consensus.cc:1261] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Refusing update from remote peer 2b88cd2cb3864003a16cf3282551d770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.161170 21062 raft_consensus.cc:1261] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Refusing update from remote peer 2b88cd2cb3864003a16cf3282551d770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.161878 21428 consensus_queue.cc:1035] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.162089 21425 consensus_queue.cc:1035] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.171231 21433 mvcc.cc:205] Tried to move back new op lower bound from 7332929692300623872 to 7332929692170584064. Current Snapshot: MvccSnapshot[applied={T|T < 7332929692300623872}]
I20260924 16:07:18.171574 21432 mvcc.cc:205] Tried to move back new op lower bound from 7332929692300623872 to 7332929692170584064. Current Snapshot: MvccSnapshot[applied={T|T < 7332929692300623872}]
I20260924 16:07:18.178118 20903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:18.178354 20903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:18.180860 20903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Sending DeleteTablet for 3 replicas of tablet c085847ffcec4763998aec3fadce8400
I20260924 16:07:18.181646 21042 tablet_service.cc:1514] Processing DeleteTablet for tablet c085847ffcec4763998aec3fadce8400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:57482
I20260924 16:07:18.181978 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet c085847ffcec4763998aec3fadce8400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51338
I20260924 16:07:18.182240 21324 tablet_service.cc:1514] Processing DeleteTablet for tablet c085847ffcec4763998aec3fadce8400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:60982
I20260924 16:07:18.182487 20903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 16:07:18.182749 21441 tablet_replica.cc:330] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: stopping tablet replica
I20260924 16:07:18.182916 21444 tablet_replica.cc:330] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: stopping tablet replica
W20260924 16:07:18.182950 20903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:18.183099 21441 raft_consensus.cc:2229] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:18.183418 21444 raft_consensus.cc:2229] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:18.183476 21441 raft_consensus.cc:2258] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.185070 21441 ts_tablet_manager.cc:1905] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.184465 21444 raft_consensus.cc:2258] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.188004 21042 tablet_service.cc:1467] Processing CreateTablet for tablet 01068a74af6841bc8a37edb658b96c61 (DEFAULT_TABLE table=test-table [id=22bacb5b88a54217bfdcd7e1205d2512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.188120 21444 ts_tablet_manager.cc:1905] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.188174 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 01068a74af6841bc8a37edb658b96c61 (DEFAULT_TABLE table=test-table [id=22bacb5b88a54217bfdcd7e1205d2512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.188298 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01068a74af6841bc8a37edb658b96c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.188462 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01068a74af6841bc8a37edb658b96c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.190400 21443 tablet_replica.cc:330] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: stopping tablet replica
I20260924 16:07:18.190737 21443 raft_consensus.cc:2229] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:18.190985 21444 ts_tablet_manager.cc:1918] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:18.191037 21418 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Bootstrap starting.
I20260924 16:07:18.191062 21443 raft_consensus.cc:2258] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.191095 21444 log.cc:1196] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/c085847ffcec4763998aec3fadce8400
I20260924 16:07:18.191217 21441 ts_tablet_manager.cc:1918] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:18.191305 21441 log.cc:1196] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/c085847ffcec4763998aec3fadce8400
I20260924 16:07:18.191627 21441 ts_tablet_manager.cc:1939] T c085847ffcec4763998aec3fadce8400 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting consensus metadata
I20260924 16:07:18.191851 21444 ts_tablet_manager.cc:1939] T c085847ffcec4763998aec3fadce8400 P 2b88cd2cb3864003a16cf3282551d770: Deleting consensus metadata
I20260924 16:07:18.192061 21418 tablet_bootstrap.cc:654] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.192781 20887 catalog_manager.cc:4841] TS 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767): tablet c085847ffcec4763998aec3fadce8400 (table test-table [id=cd153d82452b44ea8a980556e3dc0f95]) successfully deleted
I20260924 16:07:18.193048 21422 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Bootstrap starting.
I20260924 16:07:18.193631 21418 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: No bootstrap required, opened a new log
I20260924 16:07:18.193699 21418 ts_tablet_manager.cc:1397] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:18.193823 21422 tablet_bootstrap.cc:654] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.194121 21418 raft_consensus.cc:348] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.194258 21418 raft_consensus.cc:374] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.194291 21418 raft_consensus.cc:729] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.194447 21418 consensus_queue.cc:260] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.194726 21418 ts_tablet_manager.cc:1428] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:18.195343 20887 catalog_manager.cc:4841] TS 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295): tablet c085847ffcec4763998aec3fadce8400 (table test-table [id=cd153d82452b44ea8a980556e3dc0f95]) successfully deleted
I20260924 16:07:18.195604 21324 tablet_service.cc:1467] Processing CreateTablet for tablet 01068a74af6841bc8a37edb658b96c61 (DEFAULT_TABLE table=test-table [id=22bacb5b88a54217bfdcd7e1205d2512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.195861 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01068a74af6841bc8a37edb658b96c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.197983 21419 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Bootstrap starting.
I20260924 16:07:18.198482 21422 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: No bootstrap required, opened a new log
I20260924 16:07:18.198566 21422 ts_tablet_manager.cc:1397] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 16:07:18.199097 21419 tablet_bootstrap.cc:654] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.199030 21422 raft_consensus.cc:348] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.199146 21422 raft_consensus.cc:374] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.199179 21422 raft_consensus.cc:729] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213f00b51f714189922951d6479a9366, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.199322 21422 consensus_queue.cc:260] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.199615 21422 ts_tablet_manager.cc:1428] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 16:07:18.200711 21419 tablet_bootstrap.cc:492] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: No bootstrap required, opened a new log
I20260924 16:07:18.200791 21419 ts_tablet_manager.cc:1397] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:18.201308 21419 raft_consensus.cc:348] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.201387 21419 raft_consensus.cc:374] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.201408 21419 raft_consensus.cc:729] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.201519 21419 consensus_queue.cc:260] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.203032 21419 ts_tablet_manager.cc:1428] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:18.203291 21443 ts_tablet_manager.cc:1905] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.205463 21443 ts_tablet_manager.cc:1918] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:18.205540 21443 log.cc:1196] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/c085847ffcec4763998aec3fadce8400
I20260924 16:07:18.205837 21443 ts_tablet_manager.cc:1939] T c085847ffcec4763998aec3fadce8400 P 213f00b51f714189922951d6479a9366: Deleting consensus metadata
I20260924 16:07:18.207073 20887 catalog_manager.cc:4841] TS 213f00b51f714189922951d6479a9366 (127.13.8.66:38351): tablet c085847ffcec4763998aec3fadce8400 (table test-table [id=cd153d82452b44ea8a980556e3dc0f95]) successfully deleted
W20260924 16:07:18.253041 21391 tablet.cc:2369] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:18.284922 21426 raft_consensus.cc:482] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:18.285085 21426 raft_consensus.cc:504] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
W20260924 16:07:18.285673 21250 tablet.cc:2369] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:07:18.286367 21426 leader_election.cc:288] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:18.291131 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.286578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40767 (local address 127.13.8.66:53165)
0924 16:07:18.286723 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.286739 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.286842 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.287320 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.287323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.287342 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.287468 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.287476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.288452 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.288457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.289738 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.289783 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.290689 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.290695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.290792 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.290815 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.290898 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.290975 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 16:07:18.291124 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.286800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53165 (local address 127.13.8.65:40767)
0924 16:07:18.286955 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.286962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.286980 (+ 18us) server_negotiation.cc:407] Connection header received
0924 16:07:18.287039 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.287042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.287097 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.287177 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.287651 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.288338 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.289901 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.290550 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.290670 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.290742 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.290923 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.290999 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 16:07:18.291358 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.287189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:39815 (local address 127.13.8.67:33295)
0924 16:07:18.287311 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.287315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.287495 (+ 180us) server_negotiation.cc:407] Connection header received
0924 16:07:18.287592 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.287595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.287638 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.287719 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:18.287997 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.288753 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.290157 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.290634 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.290741 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.290809 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.291136 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.291226 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:07:18.291711 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.287037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.13.8.66:39815)
0924 16:07:18.287391 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.287404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.287503 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.287744 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.287747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.287765 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:18.287885 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.287891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.288889 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.288893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.290031 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.290038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.290887 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.290893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.290978 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.290997 (+ 19us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.291512 (+ 515us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.291585 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":74,"threads_started":1}
I20260924 16:07:18.292091 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01068a74af6841bc8a37edb658b96c61" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5" is_pre_election: true
I20260924 16:07:18.292275 21062 raft_consensus.cc:2454] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 0.
I20260924 16:07:18.292418 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01068a74af6841bc8a37edb658b96c61" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770" is_pre_election: true
I20260924 16:07:18.292618 21133 leader_election.cc:302] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:18.292670 21344 raft_consensus.cc:2454] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 0.
I20260924 16:07:18.292884 21426 raft_consensus.cc:2781] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:18.292977 21426 raft_consensus.cc:482] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:18.293033 21426 raft_consensus.cc:3037] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.294039 21426 raft_consensus.cc:504] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.294507 21426 leader_election.cc:288] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 election: Requested vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:18.294974 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01068a74af6841bc8a37edb658b96c61" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5"
I20260924 16:07:18.295028 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01068a74af6841bc8a37edb658b96c61" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770"
I20260924 16:07:18.295147 21062 raft_consensus.cc:3037] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.295252 21344 raft_consensus.cc:3037] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.296169 21062 raft_consensus.cc:2454] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 1.
I20260924 16:07:18.296489 21133 leader_election.cc:302] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:18.296581 21344 raft_consensus.cc:2454] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 1.
I20260924 16:07:18.296728 21426 raft_consensus.cc:2781] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:18.296970 21426 raft_consensus.cc:686] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 1 LEADER]: Becoming Leader. State: Replica: 213f00b51f714189922951d6479a9366, State: Running, Role: LEADER
I20260924 16:07:18.297232 21426 consensus_queue.cc:237] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.303005 20903 catalog_manager.cc:5495] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 reported cstate change: term changed from 0 to 1, leader changed from <none> to 213f00b51f714189922951d6479a9366 (127.13.8.66). New cstate: current_term: 1 leader_uuid: "213f00b51f714189922951d6479a9366" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } health_report { overall_health: UNKNOWN } } }
W20260924 16:07:18.317104 21109 tablet.cc:2369] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266037, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.126@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:18.363847 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.356189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33359 (local address 127.0.0.1:58444)
0924 16:07:18.356532 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.356554 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.356657 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.356886 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.356888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.357261 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:18.357355 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.357360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.358086 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.358090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.358742 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.358751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.359144 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.359146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.359207 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.359707 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:18.359718 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:18.361109 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:18.362575 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.362590 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.362595 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.362798 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.363014 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:18.363016 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:18.363017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:18.363097 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:18.363391 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:18.363396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:18.363489 (+ 93us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.363572 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.363701 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":121,"threads_started":1}
I20260924 16:07:18.363867 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.356290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.13.8.126:33359)
0924 16:07:18.356424 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.356427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.356665 (+ 238us) server_negotiation.cc:407] Connection header received
0924 16:07:18.356728 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.356730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.356770 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.356848 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.357463 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.357970 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.358848 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.359048 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.359124 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.361237 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:18.361240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:18.361242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:18.361262 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:18.362475 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.362876 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.362879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.362880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.362943 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.363166 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.363168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.363169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.363311 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:18.363377 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.363623 (+ 246us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.363756 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":54}
I20260924 16:07:18.367733 20903 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:58444:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 16:07:18.368489 20903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:18.373347 21324 tablet_service.cc:1467] Processing CreateTablet for tablet d4cadd00f2db4c89bcf58031cabf5a70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba50508c173a473d85b4d5d2fb7df585]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:07:18.373299 21042 tablet_service.cc:1467] Processing CreateTablet for tablet d4cadd00f2db4c89bcf58031cabf5a70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba50508c173a473d85b4d5d2fb7df585]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:07:18.373343 21183 tablet_service.cc:1467] Processing CreateTablet for tablet d4cadd00f2db4c89bcf58031cabf5a70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba50508c173a473d85b4d5d2fb7df585]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 16:07:18.373697 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4cadd00f2db4c89bcf58031cabf5a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.373697 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4cadd00f2db4c89bcf58031cabf5a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.373703 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4cadd00f2db4c89bcf58031cabf5a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.375936 21419 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: Bootstrap starting.
I20260924 16:07:18.376471 21422 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: Bootstrap starting.
I20260924 16:07:18.376875 21419 tablet_bootstrap.cc:654] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.377386 21422 tablet_bootstrap.cc:654] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.377629 21418 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: Bootstrap starting.
I20260924 16:07:18.378484 21418 tablet_bootstrap.cc:654] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.378652 21422 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: No bootstrap required, opened a new log
I20260924 16:07:18.378737 21422 ts_tablet_manager.cc:1397] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 16:07:18.378762 21419 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: No bootstrap required, opened a new log
I20260924 16:07:18.378832 21419 ts_tablet_manager.cc:1397] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:18.379169 21422 raft_consensus.cc:348] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.379225 21419 raft_consensus.cc:348] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.379293 21422 raft_consensus.cc:374] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.379317 21419 raft_consensus.cc:374] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.379335 21422 raft_consensus.cc:729] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213f00b51f714189922951d6479a9366, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.379354 21419 raft_consensus.cc:729] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.379477 21419 consensus_queue.cc:260] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.379477 21422 consensus_queue.cc:260] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.379751 21419 ts_tablet_manager.cc:1428] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:18.379699 21426 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.379817 21426 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.380026 21422 ts_tablet_manager.cc:1428] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:18.379957 21436 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.380096 21436 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.380301 21418 tablet_bootstrap.cc:492] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: No bootstrap required, opened a new log
I20260924 16:07:18.380385 21418 ts_tablet_manager.cc:1397] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 16:07:18.380808 21418 raft_consensus.cc:348] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.380934 21418 raft_consensus.cc:374] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.380964 21418 raft_consensus.cc:729] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.381094 21418 consensus_queue.cc:260] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.381397 21418 ts_tablet_manager.cc:1428] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:07:18.381354 21423 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.381471 21423 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.578848 21436 raft_consensus.cc:482] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:18.578980 21436 raft_consensus.cc:504] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.579465 21436 leader_election.cc:288] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 213f00b51f714189922951d6479a9366 (127.13.8.66:38351)
I20260924 16:07:18.579910 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4cadd00f2db4c89bcf58031cabf5a70" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5" is_pre_election: true
I20260924 16:07:18.579977 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4cadd00f2db4c89bcf58031cabf5a70" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366" is_pre_election: true
I20260924 16:07:18.580052 21062 raft_consensus.cc:2454] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 0.
I20260924 16:07:18.580101 21203 raft_consensus.cc:2454] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 0.
I20260924 16:07:18.580380 21274 leader_election.cc:302] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 2b88cd2cb3864003a16cf3282551d770; no voters:
I20260924 16:07:18.580539 21436 raft_consensus.cc:2781] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:18.580606 21436 raft_consensus.cc:482] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:18.580657 21436 raft_consensus.cc:3037] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.581656 21436 raft_consensus.cc:504] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.581967 21436 leader_election.cc:288] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 election: Requested vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 213f00b51f714189922951d6479a9366 (127.13.8.66:38351)
I20260924 16:07:18.582291 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4cadd00f2db4c89bcf58031cabf5a70" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5"
I20260924 16:07:18.582413 21062 raft_consensus.cc:3037] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.582435 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4cadd00f2db4c89bcf58031cabf5a70" candidate_uuid: "2b88cd2cb3864003a16cf3282551d770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366"
I20260924 16:07:18.582577 21203 raft_consensus.cc:3037] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:18.583331 21062 raft_consensus.cc:2454] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 1.
I20260924 16:07:18.583564 21203 raft_consensus.cc:2454] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b88cd2cb3864003a16cf3282551d770 in term 1.
I20260924 16:07:18.583644 21274 leader_election.cc:302] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b88cd2cb3864003a16cf3282551d770, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:18.583797 21436 raft_consensus.cc:2781] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:18.583884 21436 raft_consensus.cc:686] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [term 1 LEADER]: Becoming Leader. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Running, Role: LEADER
I20260924 16:07:18.584017 21436 consensus_queue.cc:237] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.584697 21428 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: TxnStatusTablet state changed. Reason: New leader 2b88cd2cb3864003a16cf3282551d770. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.584805 21428 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:07:18.585103 21461 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 16:07:18.585268 21461 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 16:07:18.585913 20900 catalog_manager.cc:5495] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } health_report { overall_health: HEALTHY } } }
I20260924 16:07:18.616833 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.613351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.13.8.67:33295)
0924 16:07:18.613467 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.613470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.613483 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:18.613642 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.613644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.613689 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.613759 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:18.614018 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.614565 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.615810 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.616010 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.616084 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.616421 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.616484 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.616537 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.616671 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.616727 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 16:07:18.616881 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.613211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.0.0.1:32770)
0924 16:07:18.613402 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.613418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.613521 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.613799 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.613802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.613812 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:07:18.613921 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.613926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.614672 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.614676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.615685 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.615696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.616124 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.616128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.616221 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.616292 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.616568 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.616582 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.616650 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.616732 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260924 16:07:18.619333 21062 raft_consensus.cc:1261] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Refusing update from remote peer 2b88cd2cb3864003a16cf3282551d770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.619393 21203 raft_consensus.cc:1261] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Refusing update from remote peer 2b88cd2cb3864003a16cf3282551d770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.619768 21436 consensus_queue.cc:1035] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.619913 21428 consensus_queue.cc:1035] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.621318 21426 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: TxnStatusTablet state changed. Reason: New leader 2b88cd2cb3864003a16cf3282551d770. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.621336 21423 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: TxnStatusTablet state changed. Reason: New leader 2b88cd2cb3864003a16cf3282551d770. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.621423 21426 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.621433 21423 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.623863 21436 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.623963 21436 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:07:18.624073 21436 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.624138 21436 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 2b88cd2cb3864003a16cf3282551d770: This TxnStatusTablet replica's current role is: LEADER
I20260924 16:07:18.624214 21431 mvcc.cc:205] Tried to move back new op lower bound from 7332929694180880384 to 7332929694041427968. Current Snapshot: MvccSnapshot[applied={T|T < 7332929694180880384 or (T in {7332929694180880384})}]
I20260924 16:07:18.624604 21426 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.624709 21426 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 213f00b51f714189922951d6479a9366: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.625748 21423 tablet_replica.cc:439] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b88cd2cb3864003a16cf3282551d770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } } }
I20260924 16:07:18.625847 21423 tablet_replica.cc:442] T d4cadd00f2db4c89bcf58031cabf5a70 P 5f0cd4a4430b40b48590d96cc59838c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 16:07:18.633627 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.629821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.13.8.66:38351)
0924 16:07:18.630038 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.630042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.630054 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:18.630128 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.630131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.630186 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.630312 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:18.630530 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.631254 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.632015 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.632355 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.632466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.632703 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.633253 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.633396 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.633434 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.633483 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260924 16:07:18.644475 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.640395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53229 (local address 127.13.8.67:33295)
0924 16:07:18.640498 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.640502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.640625 (+ 123us) server_negotiation.cc:407] Connection header received
0924 16:07:18.640719 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.640722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.640772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.640868 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:18.641140 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.641850 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.643229 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.643618 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.643718 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.644022 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.644107 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.644176 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.644291 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.644347 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 16:07:18.644515 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.640296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.13.8.66:53229)
0924 16:07:18.640547 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.640559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.640639 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.640883 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.640887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.640910 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:07:18.641030 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.641038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.641965 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.641968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.643123 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.643133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.643767 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.643771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.643857 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.643899 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.644187 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.644192 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.644248 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.644349 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260924 16:07:18.651325 21061 raft_consensus.cc:1261] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Refusing update from remote peer 213f00b51f714189922951d6479a9366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.651691 21344 raft_consensus.cc:1261] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Refusing update from remote peer 213f00b51f714189922951d6479a9366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:18.651759 21426 consensus_queue.cc:1035] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.652396 21426 consensus_queue.cc:1035] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:18.674613 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.670808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38351 (local address 127.13.8.67:47409)
0924 16:07:18.671082 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:18.671097 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:18.671220 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:18.671480 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:18.671483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:18.671490 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:07:18.671622 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.671630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.672596 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.672600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.673391 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:18.673401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.673854 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.673857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:18.673938 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.673995 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.674313 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.674340 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 16:07:18.674401 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 16:07:18.674477 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260924 16:07:18.674595 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.671138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:47409 (local address 127.13.8.66:38351)
0924 16:07:18.671274 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.671279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.671293 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:07:18.671333 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.671336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.671384 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.671457 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:18.671723 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.672493 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.673509 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.673746 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.673852 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.674135 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:18.674211 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:18.674308 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.674419 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.674466 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:18.702448 20900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:18.702625 20900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:18.704856 20900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Sending DeleteTablet for 3 replicas of tablet 01068a74af6841bc8a37edb658b96c61
I20260924 16:07:18.705467 21042 tablet_service.cc:1514] Processing DeleteTablet for tablet 01068a74af6841bc8a37edb658b96c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:57482
I20260924 16:07:18.705801 21477 tablet_replica.cc:330] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: stopping tablet replica
I20260924 16:07:18.705950 21477 raft_consensus.cc:2229] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:18.706063 21477 raft_consensus.cc:2258] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.706468 21324 tablet_service.cc:1514] Processing DeleteTablet for tablet 01068a74af6841bc8a37edb658b96c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:60982
I20260924 16:07:18.706753 21477 ts_tablet_manager.cc:1905] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.710070 21477 ts_tablet_manager.cc:1918] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:07:18.710145 21477 log.cc:1196] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/01068a74af6841bc8a37edb658b96c61
I20260924 16:07:18.710450 21477 ts_tablet_manager.cc:1939] T 01068a74af6841bc8a37edb658b96c61 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting consensus metadata
I20260924 16:07:18.710470 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet 01068a74af6841bc8a37edb658b96c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:18 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51338
I20260924 16:07:18.711093 20887 catalog_manager.cc:4841] TS 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767): tablet 01068a74af6841bc8a37edb658b96c61 (table test-table [id=22bacb5b88a54217bfdcd7e1205d2512]) successfully deleted
I20260924 16:07:18.713770 21480 tablet_replica.cc:330] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: stopping tablet replica
I20260924 16:07:18.713948 21480 raft_consensus.cc:2229] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:18.714367 21480 raft_consensus.cc:2258] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.714403 21478 tablet_replica.cc:330] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: stopping tablet replica
I20260924 16:07:18.714517 21478 raft_consensus.cc:2229] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:18.714640 21478 raft_consensus.cc:2258] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:18.715188 21480 ts_tablet_manager.cc:1905] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.715273 21478 ts_tablet_manager.cc:1905] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:18.717988 21478 ts_tablet_manager.cc:1918] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:07:18.718070 21478 log.cc:1196] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/01068a74af6841bc8a37edb658b96c61
I20260924 16:07:18.718357 21478 ts_tablet_manager.cc:1939] T 01068a74af6841bc8a37edb658b96c61 P 2b88cd2cb3864003a16cf3282551d770: Deleting consensus metadata
I20260924 16:07:18.718600 21480 ts_tablet_manager.cc:1918] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:07:18.718726 21480 log.cc:1196] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/01068a74af6841bc8a37edb658b96c61
I20260924 16:07:18.719012 20887 catalog_manager.cc:4841] TS 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295): tablet 01068a74af6841bc8a37edb658b96c61 (table test-table [id=22bacb5b88a54217bfdcd7e1205d2512]) successfully deleted
I20260924 16:07:18.719046 21480 ts_tablet_manager.cc:1939] T 01068a74af6841bc8a37edb658b96c61 P 213f00b51f714189922951d6479a9366: Deleting consensus metadata
I20260924 16:07:18.719794 20887 catalog_manager.cc:4841] TS 213f00b51f714189922951d6479a9366 (127.13.8.66:38351): tablet 01068a74af6841bc8a37edb658b96c61 (table test-table [id=22bacb5b88a54217bfdcd7e1205d2512]) successfully deleted
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:18.746727 20953 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:18.738724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.13.8.126:33359)
0924 16:07:18.738859 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.738863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.739184 (+ 321us) server_negotiation.cc:407] Connection header received
0924 16:07:18.739332 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.739335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.739392 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.739477 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.740396 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.741220 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.742448 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.742798 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.742899 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.746596 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58448: BlockingRecv error: recv got EOF from 127.0.0.1:58448 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 16:07:18 dist-test-slave-vj6n krb5kdc[20857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM
I20260924 16:07:18.759136 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:18.748933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.13.8.126:33359)
0924 16:07:18.749044 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:18.749048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:18.749062 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:07:18.749181 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:18.749184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:18.749235 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:18.749320 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:18.750001 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.750782 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.751896 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:18.752247 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:18.752352 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:18.755690 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:18.755694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:18.755696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:18.755724 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:18.757445 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.757925 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.757928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.757930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.758018 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:18.758336 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:18.758338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:18.758339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:18.758515 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:18.758616 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:18.758857 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 16:07:18.758949 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:18.761914 20900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:18.762408 20900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:18.767251 21324 tablet_service.cc:1467] Processing CreateTablet for tablet 4e138812e52749f283217065314f6cc0 (DEFAULT_TABLE table=test-table [id=e8017e05c8204c48bc810cd0caf93cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.767292 21042 tablet_service.cc:1467] Processing CreateTablet for tablet 4e138812e52749f283217065314f6cc0 (DEFAULT_TABLE table=test-table [id=e8017e05c8204c48bc810cd0caf93cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.767536 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e138812e52749f283217065314f6cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.767599 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e138812e52749f283217065314f6cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.769701 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 4e138812e52749f283217065314f6cc0 (DEFAULT_TABLE table=test-table [id=e8017e05c8204c48bc810cd0caf93cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:18.769954 21419 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Bootstrap starting.
I20260924 16:07:18.770015 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e138812e52749f283217065314f6cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:18.770829 21419 tablet_bootstrap.cc:654] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.772032 21419 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: No bootstrap required, opened a new log
I20260924 16:07:18.772113 21419 ts_tablet_manager.cc:1397] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:18.772368 21418 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Bootstrap starting.
I20260924 16:07:18.772522 21419 raft_consensus.cc:348] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.772626 21419 raft_consensus.cc:374] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.772657 21419 raft_consensus.cc:729] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.772779 21419 consensus_queue.cc:260] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.773022 21419 ts_tablet_manager.cc:1428] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:18.773337 21418 tablet_bootstrap.cc:654] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.773969 21422 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Bootstrap starting.
I20260924 16:07:18.774857 21422 tablet_bootstrap.cc:654] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:18.775964 21418 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: No bootstrap required, opened a new log
I20260924 16:07:18.776057 21418 ts_tablet_manager.cc:1397] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 16:07:18.776615 21418 raft_consensus.cc:348] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.776710 21418 raft_consensus.cc:374] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.776732 21418 raft_consensus.cc:729] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.776859 21418 consensus_queue.cc:260] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.777133 21418 ts_tablet_manager.cc:1428] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:07:18.777544 21422 tablet_bootstrap.cc:492] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: No bootstrap required, opened a new log
I20260924 16:07:18.777666 21422 ts_tablet_manager.cc:1397] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 16:07:18.778034 21422 raft_consensus.cc:348] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.778151 21422 raft_consensus.cc:374] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:18.778196 21422 raft_consensus.cc:729] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213f00b51f714189922951d6479a9366, State: Initialized, Role: FOLLOWER
I20260924 16:07:18.778355 21422 consensus_queue.cc:260] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:18.778620 21422 ts_tablet_manager.cc:1428] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:19.079946 21472 raft_consensus.cc:482] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:19.080087 21472 raft_consensus.cc:504] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.080559 21472 leader_election.cc:288] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:19.081116 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e138812e52749f283217065314f6cc0" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770" is_pre_election: true
I20260924 16:07:19.081122 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e138812e52749f283217065314f6cc0" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5" is_pre_election: true
I20260924 16:07:19.081269 21344 raft_consensus.cc:2454] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 0.
I20260924 16:07:19.081293 21062 raft_consensus.cc:2454] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 0.
I20260924 16:07:19.081589 21133 leader_election.cc:302] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 2b88cd2cb3864003a16cf3282551d770; no voters:
I20260924 16:07:19.081802 21472 raft_consensus.cc:2781] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:19.081864 21472 raft_consensus.cc:482] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:19.081890 21472 raft_consensus.cc:3037] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.082923 21472 raft_consensus.cc:504] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.083245 21472 leader_election.cc:288] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 election: Requested vote from peers 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:19.083629 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e138812e52749f283217065314f6cc0" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f0cd4a4430b40b48590d96cc59838c5"
I20260924 16:07:19.083758 21062 raft_consensus.cc:3037] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.083863 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e138812e52749f283217065314f6cc0" candidate_uuid: "213f00b51f714189922951d6479a9366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770"
I20260924 16:07:19.083979 21344 raft_consensus.cc:3037] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.084653 21062 raft_consensus.cc:2454] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 1.
I20260924 16:07:19.084952 21133 leader_election.cc:302] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:19.084987 21344 raft_consensus.cc:2454] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213f00b51f714189922951d6479a9366 in term 1.
I20260924 16:07:19.085201 21472 raft_consensus.cc:2781] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:19.085328 21472 raft_consensus.cc:686] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 1 LEADER]: Becoming Leader. State: Replica: 213f00b51f714189922951d6479a9366, State: Running, Role: LEADER
I20260924 16:07:19.085474 21472 consensus_queue.cc:237] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.087142 20903 catalog_manager.cc:5495] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 reported cstate change: term changed from 0 to 1, leader changed from <none> to 213f00b51f714189922951d6479a9366 (127.13.8.66). New cstate: current_term: 1 leader_uuid: "213f00b51f714189922951d6479a9366" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:19.134939 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.132298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.13.8.66:38351)
0924 16:07:19.132424 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.132428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.132441 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:19.132491 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.132492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.132537 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.132604 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:19.132834 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.133335 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.134101 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.134321 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.134394 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.134603 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.134666 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.134723 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.134796 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.134838 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:19.136989 21344 raft_consensus.cc:1261] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Refusing update from remote peer 213f00b51f714189922951d6479a9366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:19.137032 21062 raft_consensus.cc:1261] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Refusing update from remote peer 213f00b51f714189922951d6479a9366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:19.137518 21449 consensus_queue.cc:1035] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:19.137734 21472 consensus_queue.cc:1035] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:19.146387 20903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:19.146515 20903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:19.148518 20903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Sending DeleteTablet for 3 replicas of tablet 4e138812e52749f283217065314f6cc0
I20260924 16:07:19.149040 21042 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e138812e52749f283217065314f6cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:57482
I20260924 16:07:19.149096 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e138812e52749f283217065314f6cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51338
I20260924 16:07:19.149277 21477 tablet_replica.cc:330] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: stopping tablet replica
I20260924 16:07:19.149287 21480 tablet_replica.cc:330] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: stopping tablet replica
I20260924 16:07:19.149299 21324 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e138812e52749f283217065314f6cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:60982
I20260924 16:07:19.149405 21477 raft_consensus.cc:2229] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:19.149432 21480 raft_consensus.cc:2229] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:19.149484 21478 tablet_replica.cc:330] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: stopping tablet replica
I20260924 16:07:19.149523 21477 raft_consensus.cc:2258] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.149591 21478 raft_consensus.cc:2229] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:19.149627 21480 raft_consensus.cc:2258] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.149726 21478 raft_consensus.cc:2258] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.150450 21477 ts_tablet_manager.cc:1905] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:19.150470 21480 ts_tablet_manager.cc:1905] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:19.150386 20903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:19.150791 20903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:19.152806 21477 ts_tablet_manager.cc:1918] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:19.152896 21477 log.cc:1196] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/4e138812e52749f283217065314f6cc0
I20260924 16:07:19.153012 21480 ts_tablet_manager.cc:1918] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:19.153095 21480 log.cc:1196] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/4e138812e52749f283217065314f6cc0
I20260924 16:07:19.153316 21480 ts_tablet_manager.cc:1939] T 4e138812e52749f283217065314f6cc0 P 213f00b51f714189922951d6479a9366: Deleting consensus metadata
I20260924 16:07:19.154335 20887 catalog_manager.cc:4841] TS 213f00b51f714189922951d6479a9366 (127.13.8.66:38351): tablet 4e138812e52749f283217065314f6cc0 (table test-table [id=e8017e05c8204c48bc810cd0caf93cea]) successfully deleted
I20260924 16:07:19.154400 21478 ts_tablet_manager.cc:1905] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:19.154736 21477 ts_tablet_manager.cc:1939] T 4e138812e52749f283217065314f6cc0 P 5f0cd4a4430b40b48590d96cc59838c5: Deleting consensus metadata
I20260924 16:07:19.155371 20887 catalog_manager.cc:4841] TS 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767): tablet 4e138812e52749f283217065314f6cc0 (table test-table [id=e8017e05c8204c48bc810cd0caf93cea]) successfully deleted
I20260924 16:07:19.156657 21478 ts_tablet_manager.cc:1918] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:19.156723 21478 log.cc:1196] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/4e138812e52749f283217065314f6cc0
I20260924 16:07:19.157009 21478 ts_tablet_manager.cc:1939] T 4e138812e52749f283217065314f6cc0 P 2b88cd2cb3864003a16cf3282551d770: Deleting consensus metadata
I20260924 16:07:19.157625 20887 catalog_manager.cc:4841] TS 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295): tablet 4e138812e52749f283217065314f6cc0 (table test-table [id=e8017e05c8204c48bc810cd0caf93cea]) successfully deleted
I20260924 16:07:19.157868 21042 tablet_service.cc:1467] Processing CreateTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba (DEFAULT_TABLE table=test-table [id=ff07cea137b64e5cb245d5feae5c6271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:19.158133 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3156341dd3e6471f87bc0c3ec90138ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:19.158143 21324 tablet_service.cc:1467] Processing CreateTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba (DEFAULT_TABLE table=test-table [id=ff07cea137b64e5cb245d5feae5c6271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:19.158425 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3156341dd3e6471f87bc0c3ec90138ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:19.159936 21418 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: Bootstrap starting.
I20260924 16:07:19.160060 21419 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Bootstrap starting.
I20260924 16:07:19.160272 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba (DEFAULT_TABLE table=test-table [id=ff07cea137b64e5cb245d5feae5c6271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:19.160503 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3156341dd3e6471f87bc0c3ec90138ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:19.160749 21418 tablet_bootstrap.cc:654] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:19.160748 21419 tablet_bootstrap.cc:654] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:19.161866 21419 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: No bootstrap required, opened a new log
I20260924 16:07:19.161972 21419 ts_tablet_manager.cc:1397] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:19.162375 21419 raft_consensus.cc:348] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.162500 21419 raft_consensus.cc:374] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:19.162541 21419 raft_consensus.cc:729] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b88cd2cb3864003a16cf3282551d770, State: Initialized, Role: FOLLOWER
I20260924 16:07:19.162676 21419 consensus_queue.cc:260] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.162844 21422 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Bootstrap starting.
I20260924 16:07:19.162974 21419 ts_tablet_manager.cc:1428] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:19.163439 21418 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: No bootstrap required, opened a new log
I20260924 16:07:19.163529 21418 ts_tablet_manager.cc:1397] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 16:07:19.163648 21422 tablet_bootstrap.cc:654] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:19.163973 21418 raft_consensus.cc:348] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.164079 21418 raft_consensus.cc:374] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:19.164113 21418 raft_consensus.cc:729] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Initialized, Role: FOLLOWER
I20260924 16:07:19.164232 21418 consensus_queue.cc:260] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.164521 21418 ts_tablet_manager.cc:1428] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:07:19.164836 21422 tablet_bootstrap.cc:492] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: No bootstrap required, opened a new log
I20260924 16:07:19.164913 21422 ts_tablet_manager.cc:1397] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:19.165270 21422 raft_consensus.cc:348] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.165342 21422 raft_consensus.cc:374] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:19.165361 21422 raft_consensus.cc:729] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213f00b51f714189922951d6479a9366, State: Initialized, Role: FOLLOWER
I20260924 16:07:19.165481 21422 consensus_queue.cc:260] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.165738 21422 ts_tablet_manager.cc:1428] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 16:07:19.168512 21423 raft_consensus.cc:482] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:19.168617 21423 raft_consensus.cc:504] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.169735 21423 leader_election.cc:288] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213f00b51f714189922951d6479a9366 (127.13.8.66:38351), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:19.173890 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.170376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:39813 (local address 127.13.8.67:33295)
0924 16:07:19.170633 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.170636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.170660 (+ 24us) server_negotiation.cc:407] Connection header received
0924 16:07:19.170864 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.170867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.170912 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.170992 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:19.171261 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.171773 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.172900 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.173356 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.173433 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.173514 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.173686 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.173769 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":129,"threads_started":1}
I20260924 16:07:19.173890 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.170333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.13.8.65:39813)
0924 16:07:19.170592 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:19.170608 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:19.170750 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:19.171009 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:19.171012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:19.171029 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:19.171159 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.171165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.171864 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.171866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.172782 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.172791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.173465 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.173470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.173563 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.173587 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:07:19.173657 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 16:07:19.173720 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260924 16:07:19.174010 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.169943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59959 (local address 127.13.8.66:38351)
0924 16:07:19.170073 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.170079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.170311 (+ 232us) server_negotiation.cc:407] Connection header received
0924 16:07:19.170506 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.170509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.170558 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.170605 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:19.171019 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.171632 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.172943 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.173390 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.173510 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.173578 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.173806 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.173876 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:07:19.174762 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3156341dd3e6471f87bc0c3ec90138ba" candidate_uuid: "5f0cd4a4430b40b48590d96cc59838c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770" is_pre_election: true
I20260924 16:07:19.174914 21344 raft_consensus.cc:2454] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f0cd4a4430b40b48590d96cc59838c5 in term 0.
I20260924 16:07:19.175148 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.169897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38351 (local address 127.13.8.65:59959)
0924 16:07:19.170235 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:19.170252 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:19.170380 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:19.170763 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:19.170770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:19.170787 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:19.170924 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.170933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.171837 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.171840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.172794 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.172805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.173555 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.173559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.173670 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.173694 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:07:19.173774 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 16:07:19.173839 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":80,"thread_start_us":81,"threads_started":1}
I20260924 16:07:19.175304 20992 leader_election.cc:302] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b88cd2cb3864003a16cf3282551d770, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:19.175586 21423 raft_consensus.cc:2781] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:19.175657 21423 raft_consensus.cc:482] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:19.175698 21423 raft_consensus.cc:3037] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.175876 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3156341dd3e6471f87bc0c3ec90138ba" candidate_uuid: "5f0cd4a4430b40b48590d96cc59838c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366" is_pre_election: true
I20260924 16:07:19.176008 21203 raft_consensus.cc:2454] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f0cd4a4430b40b48590d96cc59838c5 in term 0.
I20260924 16:07:19.176750 21423 raft_consensus.cc:504] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.177062 21423 leader_election.cc:288] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [CANDIDATE]: Term 1 election: Requested vote from peers 213f00b51f714189922951d6479a9366 (127.13.8.66:38351), 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295)
I20260924 16:07:19.177413 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3156341dd3e6471f87bc0c3ec90138ba" candidate_uuid: "5f0cd4a4430b40b48590d96cc59838c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213f00b51f714189922951d6479a9366"
I20260924 16:07:19.177551 21203 raft_consensus.cc:3037] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.177543 21344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3156341dd3e6471f87bc0c3ec90138ba" candidate_uuid: "5f0cd4a4430b40b48590d96cc59838c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b88cd2cb3864003a16cf3282551d770"
I20260924 16:07:19.177673 21344 raft_consensus.cc:3037] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:19.178609 21203 raft_consensus.cc:2454] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f0cd4a4430b40b48590d96cc59838c5 in term 1.
I20260924 16:07:19.178740 21344 raft_consensus.cc:2454] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f0cd4a4430b40b48590d96cc59838c5 in term 1.
I20260924 16:07:19.178916 20992 leader_election.cc:302] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213f00b51f714189922951d6479a9366, 5f0cd4a4430b40b48590d96cc59838c5; no voters:
I20260924 16:07:19.179086 21423 raft_consensus.cc:2781] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:19.179265 21423 raft_consensus.cc:686] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 LEADER]: Becoming Leader. State: Replica: 5f0cd4a4430b40b48590d96cc59838c5, State: Running, Role: LEADER
I20260924 16:07:19.179522 21423 consensus_queue.cc:237] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } }
I20260924 16:07:19.181718 20902 catalog_manager.cc:5495] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65). New cstate: current_term: 1 leader_uuid: "5f0cd4a4430b40b48590d96cc59838c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f0cd4a4430b40b48590d96cc59838c5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 40767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:19.199070 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.195344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.13.8.65:40767)
0924 16:07:19.195617 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.195625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.195637 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:19.195716 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.195718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.195766 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.195893 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:19.196124 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.196674 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.197504 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.197755 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.197864 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.198139 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.198670 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.198805 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.198863 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.198914 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260924 16:07:19.208712 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.205491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:54911 (local address 127.13.8.67:33295)
0924 16:07:19.205623 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.205627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.205765 (+ 138us) server_negotiation.cc:407] Connection header received
0924 16:07:19.205928 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.205931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.205982 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.206056 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:19.206311 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.206791 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.207651 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.207868 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.207974 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.208255 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.208339 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.208417 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.208505 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.208558 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:07:19.208738 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.205352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33295 (local address 127.13.8.65:54911)
0924 16:07:19.205689 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:19.205701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:19.205818 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:19.206065 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:19.206067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:19.206074 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:07:19.206187 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.206193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.206900 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.206906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.207518 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.207526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.207977 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.207980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.208073 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.208112 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.208421 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.208430 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 16:07:19.208495 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 16:07:19.208566 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":114,"threads_started":1}
I20260924 16:07:19.214473 21344 raft_consensus.cc:1261] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Refusing update from remote peer 5f0cd4a4430b40b48590d96cc59838c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:19.214519 21203 raft_consensus.cc:1261] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Refusing update from remote peer 5f0cd4a4430b40b48590d96cc59838c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:19.214919 21505 consensus_queue.cc:1035] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b88cd2cb3864003a16cf3282551d770" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:19.215101 21423 consensus_queue.cc:1035] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213f00b51f714189922951d6479a9366" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:19.233629 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.230239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:53179 (local address 127.13.8.65:40767)
0924 16:07:19.230373 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:19.230378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:19.230585 (+ 207us) server_negotiation.cc:407] Connection header received
0924 16:07:19.230726 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:19.230730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:19.230786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:19.230861 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:19.231065 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.231543 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.232394 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.232728 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.232836 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.233123 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.233190 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.233281 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:19.233393 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 16:07:19.233462 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 16:07:19.233687 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.230131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40767 (local address 127.13.8.67:53179)
0924 16:07:19.230501 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:19.230513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:19.230626 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:19.230905 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:19.230907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:19.230913 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 16:07:19.230998 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.231002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.231628 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.231630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.232263 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:19.232275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:19.232845 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:19.232852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:19.232944 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:19.232979 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 16:07:19.233299 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 16:07:19.233305 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:07:19.233370 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 16:07:19.233463 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":83,"threads_started":1}
I20260924 16:07:19.255323 20902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:19.255486 20902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:19.257658 20902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c643edeee0ed498cb00fdcc47445b386: Sending DeleteTablet for 3 replicas of tablet 3156341dd3e6471f87bc0c3ec90138ba
I20260924 16:07:19.258236 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51338
I20260924 16:07:19.258430 21480 tablet_replica.cc:330] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: stopping tablet replica
I20260924 16:07:19.258422 21042 tablet_service.cc:1514] Processing DeleteTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:57482
I20260924 16:07:19.258559 21480 raft_consensus.cc:2229] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:19.258591 21477 tablet_replica.cc:330] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5: stopping tablet replica
I20260924 16:07:19.258669 21480 raft_consensus.cc:2258] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.258697 21477 raft_consensus.cc:2229] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:19.258970 21477 raft_consensus.cc:2258] T 3156341dd3e6471f87bc0c3ec90138ba P 5f0cd4a4430b40b48590d96cc59838c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.259405 21480 ts_tablet_manager.cc:1905] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:19.260066 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20976
I20260924 16:07:19.261381 21324 tablet_service.cc:1514] Processing DeleteTablet for tablet 3156341dd3e6471f87bc0c3ec90138ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:19 UTC) from {username='oryx', principal='oryx/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:60982
I20260924 16:07:19.261574 21478 tablet_replica.cc:330] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: stopping tablet replica
I20260924 16:07:19.261719 21478 raft_consensus.cc:2229] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:19.261844 21478 raft_consensus.cc:2258] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:19.262617 21480 ts_tablet_manager.cc:1918] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:07:19.262701 21480 log.cc:1196] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/3156341dd3e6471f87bc0c3ec90138ba
I20260924 16:07:19.262989 21480 ts_tablet_manager.cc:1939] T 3156341dd3e6471f87bc0c3ec90138ba P 213f00b51f714189922951d6479a9366: Deleting consensus metadata
I20260924 16:07:19.263293 21478 ts_tablet_manager.cc:1905] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:19.263687 20887 catalog_manager.cc:4841] TS 213f00b51f714189922951d6479a9366 (127.13.8.66:38351): tablet 3156341dd3e6471f87bc0c3ec90138ba (table test-table [id=ff07cea137b64e5cb245d5feae5c6271]) successfully deleted
I20260924 16:07:19.265316 21478 ts_tablet_manager.cc:1918] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 16:07:19.265378 21478 log.cc:1196] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/3156341dd3e6471f87bc0c3ec90138ba
I20260924 16:07:19.265610 21478 ts_tablet_manager.cc:1939] T 3156341dd3e6471f87bc0c3ec90138ba P 2b88cd2cb3864003a16cf3282551d770: Deleting consensus metadata
I20260924 16:07:19.266264 20887 catalog_manager.cc:4841] TS 2b88cd2cb3864003a16cf3282551d770 (127.13.8.67:33295): tablet 3156341dd3e6471f87bc0c3ec90138ba (table test-table [id=ff07cea137b64e5cb245d5feae5c6271]) successfully deleted
W20260924 16:07:19.268450 20887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:40767 (error 108)
I20260924 16:07:19.268597 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21117
I20260924 16:07:19.271754 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:19.271393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:40767 (local address 127.0.0.1:57496)
0924 16:07:19.271601 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:19.271676 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260924 16:07:19.271885 20887 catalog_manager.cc:4584] TS 5f0cd4a4430b40b48590d96cc59838c5 (127.13.8.65:40767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3156341dd3e6471f87bc0c3ec90138ba: Network error: Client connection negotiation failed: client connection to 127.13.8.65:40767: connect: Connection refused (error 111)
I20260924 16:07:19.276057 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21258
I20260924 16:07:19.282971 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 20871
2026-09-24T16:07:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3322 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:19 dist-test-slave-vj6n krb5kdc[21524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:19 dist-test-slave-vj6n krb5kdc[21524](info): set up 2 sockets
Sep 24 16:07:19 dist-test-slave-vj6n krb5kdc[21524](info): commencing operation
krb5kdc: starting...
W20260924 16:07:20.316004 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:20 dist-test-slave-vj6n krb5kdc[21524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:20.437378 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35879
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:20.552611 21537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:20.552886 21537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:20.552924 21537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:20.556939 21537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:20.556989 21537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:20.557008 21537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:20.557024 21537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:20.557068 21537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:20.562350 21537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:20.563400 21537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:20.564435 21537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:20.572939 21543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:20.573004 21545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:20.572952 21542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:20.573077 21537 server_base.cc:1034] running on GCE node
I20260924 16:07:20.573760 21537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:20.574916 21537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:20.576076 21537 hybrid_clock.cc:648] HybridClock initialized: now 1790266040576050 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T16:07:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1301 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:20 dist-test-slave-vj6n krb5kdc[21550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:20 dist-test-slave-vj6n krb5kdc[21550](info): set up 2 sockets
Sep 24 16:07:20 dist-test-slave-vj6n krb5kdc[21550](info): commencing operation
krb5kdc: starting...
W20260924 16:07:21.640086 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:21 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:21.765829 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43869
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:43869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:21.884138 21563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:21.884384 21563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:21.884423 21563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:21.888339 21563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:21.888391 21563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:21.888410 21563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:21.888427 21563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:21.888441 21563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:21.893847 21563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:43869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:21.894953 21563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:21.895989 21563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:21.904440 21568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:21.904464 21569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:21.904464 21571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:21.904533 21563 server_base.cc:1034] running on GCE node
I20260924 16:07:21.905185 21563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:21.906370 21563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:21.907553 21563 hybrid_clock.cc:648] HybridClock initialized: now 1790266041907534 us; error 30 us; skew 500 ppm
Sep 24 16:07:21 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266041, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:21.910316 21563 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:21.911242 21563 webserver.cc:460] Webserver started at http://127.13.8.126:45715/ using document root <none> and password file <none>
I20260924 16:07:21.911808 21563 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:21.911897 21563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:21.912134 21563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:21.914177 21563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "bbc9d84b2af244938c25a07817c2d15b"
format_stamp: "Formatted at 2026-09-24 16:07:21 on dist-test-slave-vj6n"
server_key: "fcc2c4c1a0d37135ceec993fcc2d9537"
server_key_iv: "1768fdab4475083654751741a0fa5cd3"
server_key_version: "encryptionkey@0"
I20260924 16:07:21.914841 21563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "bbc9d84b2af244938c25a07817c2d15b"
format_stamp: "Formatted at 2026-09-24 16:07:21 on dist-test-slave-vj6n"
server_key: "fcc2c4c1a0d37135ceec993fcc2d9537"
server_key_iv: "1768fdab4475083654751741a0fa5cd3"
server_key_version: "encryptionkey@0"
I20260924 16:07:21.918713 21563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:07:21.921211 21578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:21.922523 21563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:21.922695 21563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "bbc9d84b2af244938c25a07817c2d15b"
format_stamp: "Formatted at 2026-09-24 16:07:21 on dist-test-slave-vj6n"
server_key: "fcc2c4c1a0d37135ceec993fcc2d9537"
server_key_iv: "1768fdab4475083654751741a0fa5cd3"
server_key_version: "encryptionkey@0"
I20260924 16:07:21.922816 21563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:21.974601 21563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:21.977445 21563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:21.977663 21563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:21.984401 21563 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:43869
I20260924 16:07:21.984414 21631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:43869 every 8 connection(s)
I20260924 16:07:21.985337 21563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:21.988034 21632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:21.993291 21632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Bootstrap starting.
I20260924 16:07:21.994129 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21563
I20260924 16:07:21.994285 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:21.994612 13345 external_mini_cluster.cc:1428] Setting key d6e8eeeb8af95b1fe4c6b315e607bf1d
I20260924 16:07:21.996297 21632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:21.997032 21632 log.cc:824] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:21.999195 21632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: No bootstrap required, opened a new log
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:22.003829 21632 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:22.004073 21632 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:22.004137 21632 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc9d84b2af244938c25a07817c2d15b, State: Initialized, Role: FOLLOWER
I20260924 16:07:22.004690 21632 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:22.004891 21632 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:22.004966 21632 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:22.005069 21632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:22.005939 21632 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:22.006346 21632 leader_election.cc:302] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbc9d84b2af244938c25a07817c2d15b; no voters:
I20260924 16:07:22.006678 21632 leader_election.cc:288] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:22.006774 21637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:22.007025 21637 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 LEADER]: Becoming Leader. State: Replica: bbc9d84b2af244938c25a07817c2d15b, State: Running, Role: LEADER
I20260924 16:07:22.007369 21637 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:22.008087 21632 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:22.009641 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:21.995740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.13.8.126:43869)
0924 16:07:21.996165 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:21.996179 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:21.996215 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:07:21.996898 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:21.996916 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:21.997296 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:21.997684 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:21.998522 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:21.999597 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.000769 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.001221 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.001337 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.004181 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:22.004187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:22.004209 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:22.004244 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:22.006196 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.006755 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.006763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.006769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.006939 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.007278 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.007281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.007283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.007804 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:22.008007 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:22.008324 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 16:07:22.008663 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260924 16:07:22.011160 21639 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbc9d84b2af244938c25a07817c2d15b. Latest consensus state: current_term: 1 leader_uuid: "bbc9d84b2af244938c25a07817c2d15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } } }
I20260924 16:07:22.011279 21639 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:22.012323 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbc9d84b2af244938c25a07817c2d15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } } }
I20260924 16:07:22.012444 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: This master's current role is: LEADER
W20260924 16:07:22.013599 21652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:07:22.013674 21652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:07:22.013763 21654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:22.014591 21654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:22.020826 21654 catalog_manager.cc:1347] Generated new cluster ID: 4819abc47ddf4c3fa56f5fa2308f1a54
I20260924 16:07:22.020948 21654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:22.035216 21654 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:22.036139 21654 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:22.044392 21654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Generated new TSK 0
I20260924 16:07:22.045050 21654 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:22.078888 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:22.190173 21659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:22.190497 21659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:22.190573 21659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:22.194635 21659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:22.194687 21659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:22.194777 21659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:22.200337 21659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:22.201795 21659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:22.203271 21659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:07:22.214883 21659 server_base.cc:1034] running on GCE node
W20260924 16:07:22.214854 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.214849 21667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.214859 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:22.215576 21659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:22.216287 21659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:22.217495 21659 hybrid_clock.cc:648] HybridClock initialized: now 1790266042217476 us; error 30 us; skew 500 ppm
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:22.220502 21659 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:22.221329 21659 webserver.cc:460] Webserver started at http://127.13.8.65:38087/ using document root <none> and password file <none>
I20260924 16:07:22.221861 21659 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:22.221918 21659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:22.222092 21659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:22.224201 21659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "14f0ef666592429c94da2e5ede4b5bd7"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "9b2546ff65d17f9ca6be6ab8b7bc5e2a"
server_key_iv: "3089b7ee44fa48f1fa85aa529d5c7604"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.224711 21659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "14f0ef666592429c94da2e5ede4b5bd7"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "9b2546ff65d17f9ca6be6ab8b7bc5e2a"
server_key_iv: "3089b7ee44fa48f1fa85aa529d5c7604"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.228652 21659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260924 16:07:22.231146 21674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.232381 21659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260924 16:07:22.232471 21659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "14f0ef666592429c94da2e5ede4b5bd7"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "9b2546ff65d17f9ca6be6ab8b7bc5e2a"
server_key_iv: "3089b7ee44fa48f1fa85aa529d5c7604"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.232556 21659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:22.247454 21659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:22.250808 21659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:22.250991 21659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:22.251567 21659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:22.252374 21659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:22.252441 21659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.252499 21659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:22.252522 21659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.261886 21659 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43041
I20260924 16:07:22.261904 21788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43041 every 8 connection(s)
I20260924 16:07:22.262880 21659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:22.265129 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21659
I20260924 16:07:22.265223 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:22.265522 13345 external_mini_cluster.cc:1428] Setting key b10f6cd54ffb55b68c9440929d967400
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:22.277313 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.264713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:56255 (local address 127.13.8.126:43869)
0924 16:07:22.264844 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:22.264847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:22.266078 (+ 1231us) server_negotiation.cc:407] Connection header received
0924 16:07:22.266937 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:22.266940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:22.266988 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:22.267089 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:22.268196 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.269073 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.270500 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.270930 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.271062 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.273785 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:22.273790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:22.273792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:22.273823 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:22.275487 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.275934 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.275936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.275937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.276037 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.276425 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.276427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.276428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.276587 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:22.276661 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:22.277100 (+ 439us) server_negotiation.cc:299] Negotiation successful
0924 16:07:22.277194 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:22.278532 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.265020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:56255)
0924 16:07:22.265725 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:22.265776 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:22.266711 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:22.267264 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:22.267271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:22.267614 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:22.268058 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.268070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.269231 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.269234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.270350 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.270361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.271151 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.271155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.271316 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.272013 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:22.272033 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:22.273611 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:22.275590 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.275594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.275606 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.275844 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.276151 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.276157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.276162 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.276340 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.276675 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:22.276681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:22.276901 (+ 220us) client_negotiation.cc:769] Sending connection context
0924 16:07:22.277053 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 16:07:22.277421 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":135,"threads_started":1}
I20260924 16:07:22.279978 21789 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43869
I20260924 16:07:22.280217 21789 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:22.280691 21789 heartbeater.cc:511] Master 127.13.8.126:43869 requested a full tablet report, sending...
I20260924 16:07:22.282370 21593 ts_manager.cc:194] Registered new tserver with Master: 14f0ef666592429c94da2e5ede4b5bd7 (127.13.8.65:43041)
I20260924 16:07:22.284073 21593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:56255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:22.293852 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:22.403345 21794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:22.403635 21794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:22.403676 21794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:22.407440 21794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:22.407490 21794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:22.407547 21794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:22.413089 21794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:22.414060 21794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:22.415014 21794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:22.424726 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.424690 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.424695 21803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:22.424914 21794 server_base.cc:1034] running on GCE node
I20260924 16:07:22.425328 21794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:22.425944 21794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:22.427105 21794 hybrid_clock.cc:648] HybridClock initialized: now 1790266042427067 us; error 46 us; skew 500 ppm
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:22.429958 21794 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:22.430850 21794 webserver.cc:460] Webserver started at http://127.13.8.66:39247/ using document root <none> and password file <none>
I20260924 16:07:22.431396 21794 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:22.431454 21794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:22.431710 21794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:22.433691 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "f07a279a2b424317a6e145ac2eb03f61"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "b9b75eb5827ea4c4ae7bbd577b802a7c"
server_key_iv: "61c808ba73e9a4fe6c0c7fe7666757b4"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.434289 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "f07a279a2b424317a6e145ac2eb03f61"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "b9b75eb5827ea4c4ae7bbd577b802a7c"
server_key_iv: "61c808ba73e9a4fe6c0c7fe7666757b4"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.437978 21794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:07:22.440425 21810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.441623 21794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:22.441742 21794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "f07a279a2b424317a6e145ac2eb03f61"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "b9b75eb5827ea4c4ae7bbd577b802a7c"
server_key_iv: "61c808ba73e9a4fe6c0c7fe7666757b4"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.441843 21794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:22.472553 21794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:22.475907 21794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:22.476110 21794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:22.476956 21794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:22.477785 21794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:22.477856 21794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.477921 21794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:22.477949 21794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.486429 21794 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:36471
I20260924 16:07:22.486462 21923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:36471 every 8 connection(s)
I20260924 16:07:22.487305 21794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:22.489899 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21794
I20260924 16:07:22.490046 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:22.490425 13345 external_mini_cluster.cc:1428] Setting key 939d749fa8548eee8451977d51aa0056
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:22.502430 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.488982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:58025 (local address 127.13.8.126:43869)
0924 16:07:22.489122 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:22.489127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:22.490379 (+ 1252us) server_negotiation.cc:407] Connection header received
0924 16:07:22.491490 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:22.491494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:22.491547 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:22.491614 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:22.493268 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.493995 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.495268 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.495687 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.495817 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.498637 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:22.498641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:22.498644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:22.498675 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:22.500473 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.501011 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.501013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.501015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.501089 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.501403 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.501406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.501408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.501605 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:22.501682 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:22.502157 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 16:07:22.502309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:07:22.503593 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.489347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:58025)
0924 16:07:22.490179 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:22.490266 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:22.491276 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:22.491798 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:22.491827 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:22.492325 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:22.493080 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.493112 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.494137 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.494140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.495138 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.495148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.495838 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.495841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.495981 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.496690 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:22.496706 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:22.498465 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:22.500632 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.500637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.500650 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.500899 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.501169 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.501172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.501173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.501299 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.501701 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:22.501706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:22.501935 (+ 229us) client_negotiation.cc:769] Sending connection context
0924 16:07:22.502109 (+ 174us) client_negotiation.cc:240] Negotiation successful
0924 16:07:22.502509 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":116,"threads_started":1}
I20260924 16:07:22.504794 21924 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43869
I20260924 16:07:22.505050 21924 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:22.505537 21924 heartbeater.cc:511] Master 127.13.8.126:43869 requested a full tablet report, sending...
I20260924 16:07:22.506570 21593 ts_manager.cc:194] Registered new tserver with Master: f07a279a2b424317a6e145ac2eb03f61 (127.13.8.66:36471)
I20260924 16:07:22.507144 21593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:58025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:22.518870 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:22.632426 21929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:22.632745 21929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:22.632818 21929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:22.636989 21929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:22.637068 21929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:22.637156 21929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:22.642623 21929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:22.643752 21929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:22.644858 21929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:22.654799 21935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.654834 21934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.654830 21938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:22.654982 21929 server_base.cc:1034] running on GCE node
I20260924 16:07:22.655445 21929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:22.656148 21929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:22.657338 21929 hybrid_clock.cc:648] HybridClock initialized: now 1790266042657321 us; error 28 us; skew 500 ppm
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:22.660224 21929 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:22.661130 21929 webserver.cc:460] Webserver started at http://127.13.8.67:45689/ using document root <none> and password file <none>
I20260924 16:07:22.661722 21929 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:22.661830 21929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:22.662072 21929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:22.664311 21929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "0048d415de9a4cdb98cf6c8770b0f052"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "77d0c503596a69230bf701797eccda34"
server_key_iv: "2180cd27188b15c4f167dc4ff29614f9"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.664868 21929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "0048d415de9a4cdb98cf6c8770b0f052"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "77d0c503596a69230bf701797eccda34"
server_key_iv: "2180cd27188b15c4f167dc4ff29614f9"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.668670 21929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260924 16:07:22.671159 21945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.672371 21929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:22.672511 21929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "0048d415de9a4cdb98cf6c8770b0f052"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "77d0c503596a69230bf701797eccda34"
server_key_iv: "2180cd27188b15c4f167dc4ff29614f9"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.672634 21929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:22.686115 21929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:22.689482 21929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:22.689700 21929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:22.690357 21929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:22.691274 21929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:22.691345 21929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.691418 21929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:22.691461 21929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.700174 21929 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43243
I20260924 16:07:22.700194 22058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43243 every 8 connection(s)
I20260924 16:07:22.701061 21929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:22.704639 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 21929
I20260924 16:07:22.704792 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:22.705085 13345 external_mini_cluster.cc:1428] Setting key 5dfaef297340430921dd2b5354e6f01e
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:22.714987 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.702933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:47685 (local address 127.13.8.126:43869)
0924 16:07:22.703110 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:22.703115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:22.704236 (+ 1121us) server_negotiation.cc:407] Connection header received
0924 16:07:22.705243 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:22.705247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:22.705304 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:22.705403 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:22.706836 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.707496 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.708982 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.709295 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.709377 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.711932 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:22.711935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:22.711937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:22.711975 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:22.713184 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.713645 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.713648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.713649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.713718 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:22.714003 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:22.714005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:22.714007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:22.714174 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:22.714333 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:22.714781 (+ 448us) server_negotiation.cc:299] Negotiation successful
0924 16:07:22.714873 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 16:07:22.715833 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:22.703344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:47685)
0924 16:07:22.704015 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:22.704055 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:22.704993 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:22.705640 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:22.705653 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:22.706057 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:22.706707 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.706721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.707635 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.707637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.708811 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:22.708837 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:22.709426 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:22.709429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:22.709600 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:22.710316 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:22.710332 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:22.711780 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:22.713303 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.713308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.713320 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.713561 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.713798 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:22.713800 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:22.713801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:22.713926 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:22.714347 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:22.714351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:22.714571 (+ 220us) client_negotiation.cc:769] Sending connection context
0924 16:07:22.714728 (+ 157us) client_negotiation.cc:240] Negotiation successful
0924 16:07:22.714961 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":106,"threads_started":1}
I20260924 16:07:22.717015 22059 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43869
I20260924 16:07:22.717224 22059 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:22.717772 22059 heartbeater.cc:511] Master 127.13.8.126:43869 requested a full tablet report, sending...
I20260924 16:07:22.719034 21595 ts_manager.cc:194] Registered new tserver with Master: 0048d415de9a4cdb98cf6c8770b0f052 (127.13.8.67:43243)
I20260924 16:07:22.719619 21595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:47685
I20260924 16:07:22.731118 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
Entry for principal oryx/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab.
WARNING: no policy specified for oryx/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
Entry for principal oryx/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab.
WARNING: no policy specified for oryx/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
Entry for principal oryx/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab.
I20260924 16:07:22.801057 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21659
I20260924 16:07:22.807500 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21794
I20260924 16:07:22.813705 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21929
I20260924 16:07:22.819286 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 21563
I20260924 16:07:22.825388 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43869
--webserver_interface=127.13.8.126
--webserver_port=45715
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:43869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:22.935791 22068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:22.936163 22068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:22.936266 22068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:22.940295 22068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:22.940385 22068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:22.940424 22068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:22.940474 22068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:22.940503 22068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:22.945811 22068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:43869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=45715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:22.946954 22068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:22.948006 22068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:22.956456 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:22.956545 22076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:22.956593 22068 server_base.cc:1034] running on GCE node
W20260924 16:07:22.956794 22074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:22.957319 22068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:22.958531 22068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:22.959728 22068 hybrid_clock.cc:648] HybridClock initialized: now 1790266042959702 us; error 36 us; skew 500 ppm
Sep 24 16:07:22 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:22.962438 22068 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:22.963385 22068 webserver.cc:460] Webserver started at http://127.13.8.126:45715/ using document root <none> and password file <none>
I20260924 16:07:22.963948 22068 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:22.964084 22068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:22.968449 22068 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260924 16:07:22.970803 22084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:22.972121 22068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:07:22.972276 22068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "bbc9d84b2af244938c25a07817c2d15b"
format_stamp: "Formatted at 2026-09-24 16:07:21 on dist-test-slave-vj6n"
server_key: "fcc2c4c1a0d37135ceec993fcc2d9537"
server_key_iv: "1768fdab4475083654751741a0fa5cd3"
server_key_version: "encryptionkey@0"
I20260924 16:07:22.972841 22068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:23.001881 22068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:23.006438 22068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:23.006711 22068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:23.015547 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:43869 every 8 connection(s)
I20260924 16:07:23.015555 22068 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:43869
I20260924 16:07:23.016856 22068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:23.021755 22137 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:07:23.021768 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22068
I20260924 16:07:23.022459 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:43041
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:43869
--webserver_port=38087
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:07:23.024168 22137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Bootstrap starting.
I20260924 16:07:23.047217 22137 log.cc:824] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:23.055879 22137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:07:23.056316 22137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Bootstrap complete.
I20260924 16:07:23.061973 22137 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:23.062518 22137 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc9d84b2af244938c25a07817c2d15b, State: Initialized, Role: FOLLOWER
I20260924 16:07:23.063095 22137 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:23.063239 22137 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:23.063308 22137 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:23.063401 22137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:07:23.065328 22137 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:23.065680 22137 leader_election.cc:302] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbc9d84b2af244938c25a07817c2d15b; no voters:
I20260924 16:07:23.066032 22137 leader_election.cc:288] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 16:07:23.066114 22141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 2 FOLLOWER]: Leader election won for term 2
I20260924 16:07:23.066375 22141 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [term 2 LEADER]: Becoming Leader. State: Replica: bbc9d84b2af244938c25a07817c2d15b, State: Running, Role: LEADER
I20260924 16:07:23.066754 22141 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } }
I20260924 16:07:23.067334 22137 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:23.068122 22142 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bbc9d84b2af244938c25a07817c2d15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } } }
I20260924 16:07:23.068262 22142 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:23.068938 22143 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbc9d84b2af244938c25a07817c2d15b. Latest consensus state: current_term: 2 leader_uuid: "bbc9d84b2af244938c25a07817c2d15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc9d84b2af244938c25a07817c2d15b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43869 } } }
I20260924 16:07:23.069083 22143 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:23.069234 22150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:23.072678 22150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:23.073341 22150 catalog_manager.cc:1259] Loaded cluster ID: 4819abc47ddf4c3fa56f5fa2308f1a54
I20260924 16:07:23.073403 22150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:23.075768 22150 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:23.076476 22150 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbc9d84b2af244938c25a07817c2d15b: Loaded TSK: 0
I20260924 16:07:23.077687 22150 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:07:23.149158 22139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:23.149431 22139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:23.149504 22139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:23.153514 22139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:23.153596 22139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:23.153685 22139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:23.158812 22139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.65.keytab
--principal=oryx/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:43041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=38087
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:23.159790 22139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:23.160789 22139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:23.171185 22163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.171298 22166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.171329 22164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:23.171634 22139 server_base.cc:1034] running on GCE node
I20260924 16:07:23.172060 22139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:23.172772 22139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:23.173996 22139 hybrid_clock.cc:648] HybridClock initialized: now 1790266043173945 us; error 46 us; skew 500 ppm
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266043, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:23.176955 22139 init.cc:377] Logged in from keytab as oryx/127.13.8.65@KRBTEST.COM (short username oryx)
I20260924 16:07:23.177999 22139 webserver.cc:460] Webserver started at http://127.13.8.65:38087/ using document root <none> and password file <none>
I20260924 16:07:23.178639 22139 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:23.178755 22139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:23.182963 22139 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:07:23.185046 22173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.186424 22139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:23.186553 22139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "14f0ef666592429c94da2e5ede4b5bd7"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "9b2546ff65d17f9ca6be6ab8b7bc5e2a"
server_key_iv: "3089b7ee44fa48f1fa85aa529d5c7604"
server_key_version: "encryptionkey@0"
I20260924 16:07:23.187104 22139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:23.205246 22139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:23.208516 22139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:23.208729 22139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:23.209323 22139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:23.210233 22139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:23.210279 22139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.210371 22139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:23.210400 22139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.219255 22139 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43041
I20260924 16:07:23.219291 22287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43041 every 8 connection(s)
I20260924 16:07:23.220286 22139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.230252 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22139
I20260924 16:07:23.230683 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:36471
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:43869
--webserver_port=39247
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:23.232545 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.222318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36787 (local address 127.13.8.126:43869)
0924 16:07:23.222774 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.222794 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.223294 (+ 500us) server_negotiation.cc:407] Connection header received
0924 16:07:23.224256 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.224267 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.224539 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.224813 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.225872 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.226848 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.227926 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.228367 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.228486 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.231891 (+ 3405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:36787: BlockingRecv error: recv got EOF from 127.13.8.65:36787 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":94,"threads_started":1}
I20260924 16:07:23.232565 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.222423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:36787)
0924 16:07:23.223127 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.223167 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.223917 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.224913 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.224919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.225257 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.225723 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.225737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.226959 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.226961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.227824 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.227833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.228496 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.228499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.228645 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.229163 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.229180 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.231593 (+ 2413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":136,"threads_started":1}
W20260924 16:07:23.233299 22288 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:43869 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.8.126:43869: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.239336 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.233705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:58583)
0924 16:07:23.233894 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.233906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.233964 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.234192 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.234194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.234372 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.234458 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.234463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.235548 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.235551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.236067 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.236074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.236739 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.236741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.236805 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.237295 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.237305 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.239178 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260924 16:07:23.239518 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.233821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:58583 (local address 127.13.8.126:43869)
0924 16:07:23.233962 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.233968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.233985 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:07:23.234027 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.234030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.234086 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.234173 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.234581 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.235440 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.236180 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.236616 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.236735 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.239371 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:58583: BlockingRecv error: recv got EOF from 127.13.8.65:58583 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.245282 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.239812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:43885)
0924 16:07:23.240139 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.240154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.240236 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.240491 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.240493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.240642 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.240731 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.240736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.241783 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.241785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.242407 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.242416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.242990 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.242993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.243070 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.243532 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.243543 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.245173 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":110,"threads_started":1}
W20260924 16:07:23.245410 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.239937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:43885 (local address 127.13.8.126:43869)
0924 16:07:23.240071 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.240076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.240228 (+ 152us) server_negotiation.cc:407] Connection header received
0924 16:07:23.240317 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.240323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.240381 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.240472 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.240841 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.241668 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.242522 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.242871 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.242967 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.245310 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:43885: BlockingRecv error: recv got EOF from 127.13.8.65:43885 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260924 16:07:23.245464 22288 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:07:23.344264 22292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:23.344496 22292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:23.344537 22292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:23.348423 22292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:23.348469 22292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:23.348523 22292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:23.353487 22292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.66.keytab
--principal=oryx/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:36471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=39247
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:23.354453 22292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:23.355382 22292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:23.365018 22301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.364988 22298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.365077 22299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:23.365254 22292 server_base.cc:1034] running on GCE node
I20260924 16:07:23.365653 22292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:23.366331 22292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:23.367509 22292 hybrid_clock.cc:648] HybridClock initialized: now 1790266043367490 us; error 34 us; skew 500 ppm
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266043, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:23.370404 22292 init.cc:377] Logged in from keytab as oryx/127.13.8.66@KRBTEST.COM (short username oryx)
I20260924 16:07:23.371178 22292 webserver.cc:460] Webserver started at http://127.13.8.66:39247/ using document root <none> and password file <none>
I20260924 16:07:23.371693 22292 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:23.371764 22292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:23.375451 22292 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:23.377511 22308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.378737 22292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:23.378892 22292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "f07a279a2b424317a6e145ac2eb03f61"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "b9b75eb5827ea4c4ae7bbd577b802a7c"
server_key_iv: "61c808ba73e9a4fe6c0c7fe7666757b4"
server_key_version: "encryptionkey@0"
I20260924 16:07:23.379416 22292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:23.404203 22292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:23.407549 22292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:23.407757 22292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:23.408368 22292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:23.409193 22292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:23.409238 22292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.409307 22292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:23.409350 22292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.418577 22292 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:36471
I20260924 16:07:23.418605 22423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:36471 every 8 connection(s)
I20260924 16:07:23.419566 22292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:23.427968 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22292
I20260924 16:07:23.428398 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:43243
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:43869
--webserver_port=45689
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:23.430681 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.421391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:37859 (local address 127.13.8.126:43869)
0924 16:07:23.421526 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.421530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.422586 (+ 1056us) server_negotiation.cc:407] Connection header received
0924 16:07:23.423484 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.423488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.423550 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.423652 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.424809 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.425334 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.426508 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.426743 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.426855 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.430520 (+ 3665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:37859: BlockingRecv error: recv got EOF from 127.13.8.66:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:07:23.431143 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.421776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:37859)
0924 16:07:23.422427 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.422466 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.423266 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.423843 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.423850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.424190 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.424657 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.424670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.425460 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.425464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.426374 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.426384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.426917 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.426921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.427093 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.427695 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.427712 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.430349 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":110,"threads_started":1}
W20260924 16:07:23.431718 22424 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:43869 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.8.126:43869: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.437034 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.432123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:40163)
0924 16:07:23.432229 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.432245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.432320 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.432561 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.432564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.432747 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.432831 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.432836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.433529 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.433535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.434071 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.434078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.434586 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.434589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.434678 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.435159 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.435169 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.436955 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260924 16:07:23.437284 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.432240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:40163 (local address 127.13.8.126:43869)
0924 16:07:23.432358 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.432365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.432380 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:23.432425 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.432427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.432461 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.432547 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.432942 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.433425 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.434208 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.434458 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.434583 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.437137 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:40163: BlockingRecv error: recv got EOF from 127.13.8.66:40163 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.442369 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.437778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:56969)
0924 16:07:23.437883 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.437899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.437967 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.438193 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.438195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.438365 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.438427 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.438432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.439099 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.439106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.439606 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.439614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.440029 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.440032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.440108 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.440541 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.440552 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.442247 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 16:07:23.442477 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.437889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:56969 (local address 127.13.8.126:43869)
0924 16:07:23.437995 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.438001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.438025 (+ 24us) server_negotiation.cc:407] Connection header received
0924 16:07:23.438065 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.438067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.438098 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.438180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.438536 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.438997 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.439720 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.439923 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.440028 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.442359 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:56969: BlockingRecv error: recv got EOF from 127.13.8.66:56969 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260924 16:07:23.442636 22424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:07:23.543126 22427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:23.543365 22427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:23.543404 22427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:23.547472 22427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:23.547528 22427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:23.547606 22427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:23.552569 22427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/krb5kdc/oryx_127.13.8.67.keytab
--principal=oryx/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:43243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=45689
--tserver_master_addrs=127.13.8.126:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:23.553459 22427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:23.554476 22427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:23.563740 22432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.563743 22435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:23.563881 22433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:23.564119 22427 server_base.cc:1034] running on GCE node
I20260924 16:07:23.564567 22427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:23.565183 22427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:23.566290 22427 hybrid_clock.cc:648] HybridClock initialized: now 1790266043566252 us; error 42 us; skew 500 ppm
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266043, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:23.569073 22427 init.cc:377] Logged in from keytab as oryx/127.13.8.67@KRBTEST.COM (short username oryx)
I20260924 16:07:23.569955 22427 webserver.cc:460] Webserver started at http://127.13.8.67:45689/ using document root <none> and password file <none>
I20260924 16:07:23.570573 22427 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:23.570681 22427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:23.574496 22427 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260924 16:07:23.576478 22442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.577728 22427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:23.577857 22427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "0048d415de9a4cdb98cf6c8770b0f052"
format_stamp: "Formatted at 2026-09-24 16:07:22 on dist-test-slave-vj6n"
server_key: "77d0c503596a69230bf701797eccda34"
server_key_iv: "2180cd27188b15c4f167dc4ff29614f9"
server_key_version: "encryptionkey@0"
I20260924 16:07:23.578435 22427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:23.592123 22427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:23.594905 22427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:23.595088 22427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:23.595639 22427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:23.596477 22427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:23.596547 22427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.596616 22427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:23.596659 22427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:23.605115 22427 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43243
I20260924 16:07:23.605142 22557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43243 every 8 connection(s)
I20260924 16:07:23.606069 22427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.615186 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22427
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:23.616844 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.607856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:60005 (local address 127.13.8.126:43869)
0924 16:07:23.608003 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.608007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.609026 (+ 1019us) server_negotiation.cc:407] Connection header received
0924 16:07:23.609897 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.609900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.609956 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.610064 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.611232 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.611808 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.612929 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.613163 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.613276 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.616751 (+ 3475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:60005: BlockingRecv error: recv got EOF from 127.13.8.67:60005 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:07:23.617477 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.608267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:60005)
0924 16:07:23.608890 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.608928 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.609699 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.610299 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.610308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.610647 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.611086 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.611099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.611931 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.611935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.612803 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.612812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.613312 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.613316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.613488 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.614058 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.614074 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.616654 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":110,"threads_started":1}
W20260924 16:07:23.618000 22558 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:43869 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.8.126:43869: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.623509 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.618398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:40771)
0924 16:07:23.618521 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.618535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.618614 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.618826 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.618828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.618943 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.619021 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.619026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.619901 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.619905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.620493 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.620501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.620925 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.620929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.621018 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.621440 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.621453 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.623354 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 16:07:23.623636 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.618503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:40771 (local address 127.13.8.126:43869)
0924 16:07:23.618620 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.618624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.618661 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:07:23.618694 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.618696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.618734 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.618805 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.619142 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.619807 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.620609 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.620821 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.620923 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.623504 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:40771: BlockingRecv error: recv got EOF from 127.13.8.67:40771 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":40}
I20260924 16:07:23.625988 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.616161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.13.8.126:43869)
0924 16:07:23.616449 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.616453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.616468 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:23.616504 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.616507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.616547 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.616639 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.617724 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.618442 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.619622 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.619921 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.620035 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.621387 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:23.621395 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:23.621414 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:23.621441 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:23.623380 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:23.623991 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:23.623996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:23.624000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:23.624156 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:23.624773 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:23.624780 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:23.624784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:23.625230 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:23.625366 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:23.625642 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 16:07:23.625870 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":215,"thread_start_us":111,"threads_started":1}
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:23 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:23.629365 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:23.624103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:36479)
0924 16:07:23.624360 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:23.624374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:23.624472 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:23.624743 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:23.624749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:23.624895 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:23.624974 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.624999 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.625858 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.625861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.626489 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:23.626498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.627001 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.627004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:23.627084 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.627452 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:23.627466 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:23.629288 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200}
W20260924 16:07:23.629524 22290 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:23.624211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:36479 (local address 127.13.8.126:43869)
0924 16:07:23.624347 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:23.624351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:23.624437 (+ 86us) server_negotiation.cc:407] Connection header received
0924 16:07:23.624554 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:23.624556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:23.624612 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:23.624692 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:23.625103 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.625728 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.626590 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:23.626886 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:23.626990 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:23.629389 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:36479: BlockingRecv error: recv got EOF from 127.13.8.67:36479 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260924 16:07:23.629557 22558 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:24.252025 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:24.246095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:55827)
0924 16:07:24.246413 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:24.246428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:24.246529 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:24.246804 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:24.246812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:24.247060 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:24.247195 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.247204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.248371 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.248375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.249075 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.249087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.249510 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.249513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.249599 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.250050 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:24.250062 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:24.251894 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":68,"threads_started":1}
W20260924 16:07:24.252223 22564 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:24.246156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:55827 (local address 127.13.8.126:43869)
0924 16:07:24.246523 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:24.246526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:24.246539 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:24.246605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:24.246607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:24.246648 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:24.246778 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:24.247328 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.248247 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.249186 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.249409 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.249489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.252042 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:55827: BlockingRecv error: recv got EOF from 127.13.8.65:55827 (error 108)
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":134,"threads_started":1}
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:24.448720 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:24.443257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:55975)
0924 16:07:24.443529 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:24.443550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:24.443711 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:24.443999 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:24.444002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:24.444209 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:24.444304 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.444310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.445082 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.445087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.445721 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.445730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.446197 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.446200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.446327 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.446787 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:24.446798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:24.448587 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
W20260924 16:07:24.448877 22564 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:24.443321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:55975 (local address 127.13.8.126:43869)
0924 16:07:24.443496 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:24.443501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:24.443650 (+ 149us) server_negotiation.cc:407] Connection header received
0924 16:07:24.443831 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:24.443836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:24.443889 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:24.443977 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:24.444424 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.444967 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.445846 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.446081 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.446192 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.448723 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:55975: BlockingRecv error: recv got EOF from 127.13.8.66:55975 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:24 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:24.636014 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:24.630147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:50571)
0924 16:07:24.630468 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:24.630485 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:24.630627 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:24.630901 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:24.630904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:24.631127 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:24.631243 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.631249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.632019 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.632024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.632629 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:24.632638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.633175 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.633179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:24.633277 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.633881 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:24.633893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:24.635897 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
W20260924 16:07:24.636142 22564 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:24.630301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:50571 (local address 127.13.8.126:43869)
0924 16:07:24.630452 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:24.630456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:24.630572 (+ 116us) server_negotiation.cc:407] Connection header received
0924 16:07:24.630739 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:24.630742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:24.630795 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:24.630881 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:24.631364 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.631910 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.632754 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:24.633048 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:24.633172 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:24.636028 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:50571: BlockingRecv error: recv got EOF from 127.13.8.67:50571 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:25.258464 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:25.252879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:60503)
0924 16:07:25.253155 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:25.253170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:25.253294 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:25.253578 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:25.253580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:25.253756 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:25.253853 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.253858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.254610 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.254613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.255200 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.255208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.255647 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.255650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.255717 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.256218 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:25.256233 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:25.258322 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
W20260924 16:07:25.258575 22568 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:25.253071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:60503 (local address 127.13.8.126:43869)
0924 16:07:25.253332 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:25.253335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:25.253349 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:07:25.253391 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:25.253393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:25.253435 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:25.253520 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:25.253953 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.254513 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.255320 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.255539 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.255612 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.258451 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:60503: BlockingRecv error: recv got EOF from 127.13.8.65:60503 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:25.454610 22569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:25.449461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:48765)
0924 16:07:25.449720 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:25.449742 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:25.449838 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:25.450058 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:25.450060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:25.450272 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:25.450366 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.450372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.451045 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.451048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.451596 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.451604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.451998 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.452000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.452062 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.452531 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:25.452542 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:25.454395 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":123,"threads_started":1}
W20260924 16:07:25.454627 22568 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:25.449594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:48765 (local address 127.13.8.126:43869)
0924 16:07:25.449746 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:25.449749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:25.449791 (+ 42us) server_negotiation.cc:407] Connection header received
0924 16:07:25.449916 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:25.449918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:25.449960 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:25.450018 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:25.450457 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.450960 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.451696 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.451907 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.451979 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.454530 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:48765: BlockingRecv error: recv got EOF from 127.13.8.66:48765 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:25 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260924 16:07:25.642163 22568 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:25.636932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37313 (local address 127.13.8.126:43869)
0924 16:07:25.637084 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:25.637089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:25.637143 (+ 54us) server_negotiation.cc:407] Connection header received
0924 16:07:25.637269 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:25.637271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:25.637316 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:25.637384 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:25.637794 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.638330 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.639071 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.639275 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.639347 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.642072 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:37313: BlockingRecv error: recv got EOF from 127.13.8.67:37313 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:07:25.642163 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:25.636805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:37313)
0924 16:07:25.637086 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:25.637098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:25.637189 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:25.637420 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:25.637422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:25.637590 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:25.637686 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.637691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.638423 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.638425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.638977 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:25.638985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:25.639366 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:25.639369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:25.639429 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:25.639884 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:25.639895 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:25.641954 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":113,"threads_started":1}
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:26.269683 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:26.259326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:33283)
0924 16:07:26.259637 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:26.259651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:26.259758 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:26.259980 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:26.259982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:26.260158 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:26.260253 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.260258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.260964 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.260967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.261521 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.261530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.261930 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.261933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.261994 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.267340 (+ 5346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:26.267353 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:26.269511 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
W20260924 16:07:26.269832 22572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:26.259414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:33283 (local address 127.13.8.126:43869)
0924 16:07:26.259671 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:26.259675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:26.259705 (+ 30us) server_negotiation.cc:407] Connection header received
0924 16:07:26.259824 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:26.259826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:26.259871 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:26.259944 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:26.260353 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.260870 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.261627 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.261835 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.261907 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.269670 (+ 7763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:33283: BlockingRecv error: recv got EOF from 127.13.8.65:33283 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:26.461637 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:26.455466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:47883)
0924 16:07:26.455948 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:26.455962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:26.456193 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:26.456531 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:26.456533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:26.456729 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:26.456827 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.456832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.457671 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.457674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.458242 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.458320 (+ 78us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.458838 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.458840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.458902 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.459466 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:26.459481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:26.461442 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":125,"threads_started":1}
W20260924 16:07:26.461668 22572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:26.455525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:47883 (local address 127.13.8.126:43869)
0924 16:07:26.455697 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:26.455702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:26.456053 (+ 351us) server_negotiation.cc:407] Connection header received
0924 16:07:26.456337 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:26.456340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:26.456396 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:26.456522 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:26.456942 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.457561 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.458458 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.458731 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.458828 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.461566 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:47883: BlockingRecv error: recv got EOF from 127.13.8.66:47883 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:26 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:26.648931 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:26.642890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:52961)
0924 16:07:26.643251 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:26.643264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:26.643407 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:26.643677 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:26.643680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:26.643884 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:26.643974 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.643979 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.644721 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.644727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.645306 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:26.645314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.645791 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.645795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:26.645906 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.646642 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:26.646654 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:26.648812 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":86,"threads_started":1}
W20260924 16:07:26.649227 22572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:26.643012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:52961 (local address 127.13.8.126:43869)
0924 16:07:26.643196 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:26.643202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:26.643340 (+ 138us) server_negotiation.cc:407] Connection header received
0924 16:07:26.643513 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:26.643516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:26.643570 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:26.643654 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:26.644089 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.644612 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.645431 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:26.645652 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:26.645776 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:26.648956 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:52961: BlockingRecv error: recv got EOF from 127.13.8.67:52961 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:27.276310 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:27.270647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:36981)
0924 16:07:27.270965 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:27.270983 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:27.271088 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:27.271391 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:27.271394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:27.271606 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:27.271709 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.271715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.272521 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.272526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.273171 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.273180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.273645 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.273649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.273744 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.274351 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:27.274362 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:27.276181 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":91,"threads_started":1}
W20260924 16:07:27.276454 22576 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:27.270715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36981 (local address 127.13.8.126:43869)
0924 16:07:27.271022 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:27.271029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:27.271045 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:07:27.271214 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:27.271217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:27.271272 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:27.271366 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:27.271834 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.272401 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.273296 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.273529 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.273639 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.276316 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:36981: BlockingRecv error: recv got EOF from 127.13.8.65:36981 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:27.467913 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:27.462544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:41069)
0924 16:07:27.462812 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:27.462826 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:27.462930 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:27.463165 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:27.463167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:27.463344 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:27.463441 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.463446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.464166 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.464168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.464727 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.464737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.465141 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.465145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.465206 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.465795 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:27.465807 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:27.467743 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260924 16:07:27.467963 22576 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:27.462632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:41069 (local address 127.13.8.126:43869)
0924 16:07:27.462778 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:27.462782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:27.462899 (+ 117us) server_negotiation.cc:407] Connection header received
0924 16:07:27.463009 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:27.463011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:27.463055 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:27.463124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:27.463542 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.464058 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.464833 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.465038 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.465112 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.467859 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:41069: BlockingRecv error: recv got EOF from 127.13.8.66:41069 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:27 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:27.655326 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:27.649776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:53117)
0924 16:07:27.650098 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:27.650112 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:27.650241 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:27.650537 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:27.650540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:27.650725 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:27.650820 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.650826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.651583 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.651587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.652142 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:27.652150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.652664 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.652666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:27.652728 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.653287 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:27.653299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:27.655175 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":101,"threads_started":1}
W20260924 16:07:27.655455 22576 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:27.649850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:53117 (local address 127.13.8.126:43869)
0924 16:07:27.649992 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:27.649997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:27.650172 (+ 175us) server_negotiation.cc:407] Connection header received
0924 16:07:27.650334 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:27.650336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:27.650394 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:27.650465 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:27.650942 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.651474 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.652342 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:27.652559 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:27.652646 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:27.655311 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:53117: BlockingRecv error: recv got EOF from 127.13.8.67:53117 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:28.282385 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:28.277102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:51311)
0924 16:07:28.277420 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:28.277434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:28.277624 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:28.277862 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:28.277864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:28.278061 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:28.278154 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.278159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.278894 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.278896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.279463 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.279471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.279874 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.279877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.279936 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.280472 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:28.280484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:28.282276 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":76,"threads_started":1}
W20260924 16:07:28.282601 22580 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:28.277160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:51311 (local address 127.13.8.126:43869)
0924 16:07:28.277478 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:28.277481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:28.277547 (+ 66us) server_negotiation.cc:407] Connection header received
0924 16:07:28.277711 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:28.277713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:28.277754 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:28.277825 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:28.278295 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.278799 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.279559 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.279777 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.279849 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.282422 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:51311: BlockingRecv error: recv got EOF from 127.13.8.65:51311 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":121,"threads_started":1}
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:28.475023 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:28.468928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:34905)
0924 16:07:28.469272 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:28.469287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:28.469427 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:28.469818 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:28.469821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:28.470069 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:28.470249 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.470262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.471338 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.471341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.471946 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.471956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.472368 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.472371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.472435 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.473043 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:28.473058 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:28.474911 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":106,"threads_started":1}
W20260924 16:07:28.475276 22580 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:28.469017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:34905 (local address 127.13.8.126:43869)
0924 16:07:28.469145 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:28.469150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:28.469372 (+ 222us) server_negotiation.cc:407] Connection header received
0924 16:07:28.469570 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:28.469573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:28.469657 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:28.469764 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:28.470412 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.471078 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.472055 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.472260 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.472332 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.475039 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:34905: BlockingRecv error: recv got EOF from 127.13.8.66:34905 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:28 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:28.661731 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:28.656328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:45853)
0924 16:07:28.656613 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:28.656626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:28.656721 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:28.656963 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:28.656965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:28.657146 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:28.657238 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.657243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.657972 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.657975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.658639 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:28.658649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.659137 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.659141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:28.659210 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.659700 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:28.659712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:28.661600 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":71,"threads_started":1}
W20260924 16:07:28.661808 22580 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:28.656457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:45853 (local address 127.13.8.126:43869)
0924 16:07:28.656619 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:28.656624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:28.656696 (+ 72us) server_negotiation.cc:407] Connection header received
0924 16:07:28.656800 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:28.656802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:28.656848 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:28.656924 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:28.657336 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.657857 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.658760 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:28.658981 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:28.659081 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:28.661714 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:45853: BlockingRecv error: recv got EOF from 127.13.8.67:45853 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:29.288766 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:29.283151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:36909)
0924 16:07:29.283425 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:29.283437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:29.283530 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:29.283782 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:29.283784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:29.283952 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:29.284046 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.284051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.284760 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.284762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.285430 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.285445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.286072 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.286077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.286168 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.286817 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:29.286831 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:29.288651 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
W20260924 16:07:29.288944 22585 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:29.283197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36909 (local address 127.13.8.126:43869)
0924 16:07:29.283419 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:29.283423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:29.283500 (+ 77us) server_negotiation.cc:407] Connection header received
0924 16:07:29.283626 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:29.283628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:29.283672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:29.283747 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:29.284138 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.284661 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.285556 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.285941 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.286046 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.288777 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:36909: BlockingRecv error: recv got EOF from 127.13.8.65:36909 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:29.481462 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:29.475846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:33253)
0924 16:07:29.476144 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:29.476158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:29.476271 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:29.476530 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:29.476532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:29.476754 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:29.476867 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.476873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.477594 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.477597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.478164 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.478172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.478659 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.478662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.478747 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.479276 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:29.479288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:29.481306 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":125,"threads_started":1}
W20260924 16:07:29.481544 22585 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:29.475897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:33253 (local address 127.13.8.126:43869)
0924 16:07:29.476066 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:29.476078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:29.476240 (+ 162us) server_negotiation.cc:407] Connection header received
0924 16:07:29.476354 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:29.476356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:29.476402 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:29.476503 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:29.476964 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.477482 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.478326 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.478555 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.478642 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.481413 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:33253: BlockingRecv error: recv got EOF from 127.13.8.66:33253 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:29 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:29.668256 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:29.662755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:54627)
0924 16:07:29.663056 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:29.663070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:29.663168 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:29.663398 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:29.663400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:29.663575 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:29.663667 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.663672 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.664374 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.664376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.664921 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:29.664928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.665334 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.665337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:29.665396 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.665957 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:29.665969 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:29.668135 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":117,"threads_started":1}
W20260924 16:07:29.668378 22585 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:29.662851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:54627 (local address 127.13.8.126:43869)
0924 16:07:29.662994 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:29.662998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:29.663137 (+ 139us) server_negotiation.cc:407] Connection header received
0924 16:07:29.663251 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:29.663253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:29.663299 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:29.663358 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:29.663778 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.664284 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.665024 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:29.665239 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:29.665315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:29.668264 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:54627: BlockingRecv error: recv got EOF from 127.13.8.67:54627 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:30.295258 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:30.289654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:58073)
0924 16:07:30.289955 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:30.289968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:30.290119 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:30.290429 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:30.290431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:30.290614 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:30.290723 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.290729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.291537 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.291540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.292111 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.292120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.292525 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.292527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.292589 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.293056 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:30.293083 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:30.295092 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":93,"threads_started":1}
W20260924 16:07:30.295349 22589 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:30.289741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:58073 (local address 127.13.8.126:43869)
0924 16:07:30.289975 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:30.289979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:30.290173 (+ 194us) server_negotiation.cc:407] Connection header received
0924 16:07:30.290251 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:30.290254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:30.290306 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:30.290393 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:30.290844 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.291436 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.292213 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.292433 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.292504 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.295219 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:58073: BlockingRecv error: recv got EOF from 127.13.8.65:58073 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:30.487695 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:30.482392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:36871)
0924 16:07:30.482752 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:30.482769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:30.482877 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:30.483090 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:30.483093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:30.483270 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:30.483366 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.483371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.484188 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.484192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.484744 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.484754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.485136 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.485139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.485220 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.485718 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:30.485729 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:30.487583 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":99,"threads_started":1}
W20260924 16:07:30.487807 22589 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:30.482390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:36871 (local address 127.13.8.126:43869)
0924 16:07:30.482541 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:30.482544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:30.482827 (+ 283us) server_negotiation.cc:407] Connection header received
0924 16:07:30.482942 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:30.482944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:30.482988 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:30.483055 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:30.483459 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.484096 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.484844 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.485044 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.485116 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.487688 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:36871: BlockingRecv error: recv got EOF from 127.13.8.66:36871 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:30 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:30.674759 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:30.669206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:50205)
0924 16:07:30.669544 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:30.669559 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:30.669663 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:30.669905 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:30.669907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:30.670103 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:30.670197 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.670203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.670936 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.670939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.671545 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:30.671555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.671987 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.671990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:30.672055 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.672643 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:30.672655 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:30.674635 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
W20260924 16:07:30.674878 22589 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:30.669265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:50205 (local address 127.13.8.126:43869)
0924 16:07:30.669411 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:30.669415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:30.669628 (+ 213us) server_negotiation.cc:407] Connection header received
0924 16:07:30.669748 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:30.669750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:30.669797 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:30.669867 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:30.670309 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.670832 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.671659 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:30.671880 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:30.671955 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:30.674764 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:50205: BlockingRecv error: recv got EOF from 127.13.8.67:50205 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:31.305545 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:31.296119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:44399)
0924 16:07:31.296542 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:31.296560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:31.296673 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:31.297176 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:31.297179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:31.297429 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:31.297566 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.297578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.298373 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.298375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.298949 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.298957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.299380 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.299382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.299442 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.303562 (+ 4120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:31.303575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:31.305412 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":118,"threads_started":1}
W20260924 16:07:31.305742 22593 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:31.296302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:44399 (local address 127.13.8.126:43869)
0924 16:07:31.296603 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:31.296607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:31.296624 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:07:31.296810 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:31.296813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:31.296875 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:31.296971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:31.297699 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.298275 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.299051 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.299285 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.299361 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.305595 (+ 6234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:44399: BlockingRecv error: recv got EOF from 127.13.8.65:44399 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:31.494454 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:31.488592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:54231)
0924 16:07:31.488891 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:31.488903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:31.489001 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:31.489320 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:31.489322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:31.489493 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:31.489591 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.489596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.490631 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.490634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.491398 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.491412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.491969 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.491973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.492069 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.492551 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:31.492563 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:31.494340 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":128,"threads_started":1}
W20260924 16:07:31.494632 22593 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:31.488688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:54231 (local address 127.13.8.126:43869)
0924 16:07:31.488825 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:31.488829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:31.488968 (+ 139us) server_negotiation.cc:407] Connection header received
0924 16:07:31.489133 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:31.489136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:31.489196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:31.489292 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:31.489716 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.490525 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.491527 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.491854 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.491953 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.494470 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:54231: BlockingRecv error: recv got EOF from 127.13.8.66:54231 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:31 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:31.684542 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:31.675733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:47105)
0924 16:07:31.676092 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:31.676109 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:31.676269 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:31.676612 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:31.676618 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:31.676819 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:31.676915 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.676921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.677675 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.677677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.678250 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:31.678258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.678669 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.678672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:31.678750 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.682563 (+ 3813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:31.682574 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:31.684421 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
W20260924 16:07:31.684728 22593 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:31.675848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:47105 (local address 127.13.8.126:43869)
0924 16:07:31.675995 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:31.676002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:31.676189 (+ 187us) server_negotiation.cc:407] Connection header received
0924 16:07:31.676395 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:31.676398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:31.676460 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:31.676566 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:31.677022 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.677551 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.678353 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:31.678561 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:31.678651 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:31.684562 (+ 5911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:47105: BlockingRecv error: recv got EOF from 127.13.8.67:47105 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:32.311510 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:32.306318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:52681)
0924 16:07:32.306587 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:32.306601 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:32.306706 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:32.306944 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:32.306946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:32.307115 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:32.307207 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.307212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.307911 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.307913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.308451 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.308458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.308872 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.308874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.308946 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.309495 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:32.309507 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:32.311380 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
W20260924 16:07:32.311647 22597 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:32.306356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:52681 (local address 127.13.8.126:43869)
0924 16:07:32.306627 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:32.306630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:32.306647 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:07:32.306790 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:32.306792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:32.306834 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:32.306910 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:32.307301 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.307821 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.308543 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.308774 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.308853 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.311516 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:52681: BlockingRecv error: recv got EOF from 127.13.8.65:52681 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":68,"threads_started":1}
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:32.500775 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:32.495203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:34897)
0924 16:07:32.495479 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:32.495492 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:32.495620 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:32.495898 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:32.495900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:32.496078 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:32.496176 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.496181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.496965 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.496967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.497520 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.497528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.497956 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.497958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.498017 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.498623 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:32.498635 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:32.500609 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":106,"threads_started":1}
W20260924 16:07:32.500876 22597 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:32.495266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:34897 (local address 127.13.8.126:43869)
0924 16:07:32.495404 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:32.495407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:32.495581 (+ 174us) server_negotiation.cc:407] Connection header received
0924 16:07:32.495706 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:32.495708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:32.495757 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:32.495833 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:32.496305 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.496865 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.497630 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.497858 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.497938 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.500766 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:34897: BlockingRecv error: recv got EOF from 127.13.8.66:34897 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:32 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:32.691987 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:32.685368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:48829)
0924 16:07:32.685718 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:32.685741 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:32.685933 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:32.686359 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:32.686363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:32.686604 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:32.686712 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.686719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.687626 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.687634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.688265 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:32.688275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.688804 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.688809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:32.688933 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.689613 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:32.689625 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:32.691811 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":99,"threads_started":1}
W20260924 16:07:32.692045 22597 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:32.685493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:48829 (local address 127.13.8.126:43869)
0924 16:07:32.685698 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:32.685705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:32.685834 (+ 129us) server_negotiation.cc:407] Connection header received
0924 16:07:32.686089 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:32.686093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:32.686159 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:32.686333 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:32.686852 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.687452 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.688405 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:32.688670 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:32.688801 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:32.691937 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:48829: BlockingRecv error: recv got EOF from 127.13.8.67:48829 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:33.318287 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:33.312424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:38593)
0924 16:07:33.312737 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:33.312755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:33.312862 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:33.313220 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:33.313223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:33.313478 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:33.313662 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.313674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.314532 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.314535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.315138 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.315146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.315612 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.315615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.315685 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.316190 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:33.316201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:33.318100 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
W20260924 16:07:33.318400 22601 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:33.312529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:38593 (local address 127.13.8.126:43869)
0924 16:07:33.312807 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:33.312811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:33.312826 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:33.312981 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:33.312984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:33.313077 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:33.313166 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:33.313833 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.314431 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.315243 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.315476 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.315551 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.318263 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:38593: BlockingRecv error: recv got EOF from 127.13.8.65:38593 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":59,"threads_started":1}
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:33.506731 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:33.501546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:50369)
0924 16:07:33.501873 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:33.501886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:33.501977 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:33.502208 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:33.502210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:33.502441 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:33.502570 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.502576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.503274 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.503276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.503856 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.503864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.504258 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.504260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.504321 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.504783 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:33.504794 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:33.506613 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
W20260924 16:07:33.506865 22601 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:33.501662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:50369 (local address 127.13.8.126:43869)
0924 16:07:33.501807 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:33.501810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:33.501953 (+ 143us) server_negotiation.cc:407] Connection header received
0924 16:07:33.502057 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:33.502059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:33.502100 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:33.502170 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:33.502669 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.503170 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.503954 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.504157 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.504233 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.506747 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:50369: BlockingRecv error: recv got EOF from 127.13.8.66:50369 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:33 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:33.698181 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:33.692838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:52297)
0924 16:07:33.693114 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:33.693127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:33.693219 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:33.693453 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:33.693456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:33.693673 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:33.693766 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.693772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.694700 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.694703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.695334 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:33.695344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.695730 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.695732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:33.695791 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.696238 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:33.696249 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:33.698016 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260924 16:07:33.698246 22601 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:33.692958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:52297 (local address 127.13.8.126:43869)
0924 16:07:33.693094 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:33.693097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:33.693174 (+ 77us) server_negotiation.cc:407] Connection header received
0924 16:07:33.693282 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:33.693284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:33.693324 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:33.693391 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:33.693897 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.694589 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.695436 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:33.695636 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:33.695722 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:33.698125 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:52297: BlockingRecv error: recv got EOF from 127.13.8.67:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:34.324782 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:34.319159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:49467)
0924 16:07:34.319489 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:34.319501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:34.319600 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:34.319897 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:34.319900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:34.320098 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:34.320202 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.320208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.320930 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.320933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.321561 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.321569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.321982 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.321985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.322066 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.322728 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:34.322751 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:34.324614 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
W20260924 16:07:34.324862 22605 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:34.319192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:49467 (local address 127.13.8.126:43869)
0924 16:07:34.319464 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:34.319469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:34.319570 (+ 101us) server_negotiation.cc:407] Connection header received
0924 16:07:34.319679 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:34.319681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:34.319735 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:34.319813 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:34.320311 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.320832 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.321668 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.321885 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.321955 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.324738 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:49467: BlockingRecv error: recv got EOF from 127.13.8.65:49467 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:34.512802 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:34.507636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:33939)
0924 16:07:34.507927 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:34.507941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:34.508063 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:34.508305 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:34.508307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:34.508484 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:34.508582 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.508587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.509369 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.509371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.509925 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.509933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.510419 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.510421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.510483 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.510944 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:34.510955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:34.512679 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
W20260924 16:07:34.512970 22605 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:34.507704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:33939 (local address 127.13.8.126:43869)
0924 16:07:34.507875 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:34.507879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:34.507986 (+ 107us) server_negotiation.cc:407] Connection header received
0924 16:07:34.508153 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:34.508155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:34.508198 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:34.508265 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:34.508717 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.509254 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.510035 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.510309 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.510400 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.512812 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:33939: BlockingRecv error: recv got EOF from 127.13.8.66:33939 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:34 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:34.704035 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:34.698979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:42537)
0924 16:07:34.699236 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:34.699249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:34.699356 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:34.699563 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:34.699565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:34.699738 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:34.699832 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.699838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.700553 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.700555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.701112 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:34.701119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.701572 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.701575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:34.701655 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.702134 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:34.702145 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:34.703924 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
W20260924 16:07:34.704169 22605 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:34.699036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:42537 (local address 127.13.8.126:43869)
0924 16:07:34.699190 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:34.699194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:34.699296 (+ 102us) server_negotiation.cc:407] Connection header received
0924 16:07:34.699429 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:34.699431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:34.699474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:34.699530 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:34.699931 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.700463 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.701273 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:34.701481 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:34.701580 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:34.704054 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:42537: BlockingRecv error: recv got EOF from 127.13.8.67:42537 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:35.330626 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:35.325519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:35799)
0924 16:07:35.325814 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:35.325828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:35.325926 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:35.326179 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:35.326181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:35.326369 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:35.326463 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.326468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.327191 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.327193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.327760 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.327767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.328170 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.328172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.328233 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.328708 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:35.328720 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:35.330502 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
W20260924 16:07:35.330812 22609 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:35.325648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:35799 (local address 127.13.8.126:43869)
0924 16:07:35.325898 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:35.325902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:35.325915 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:35.326006 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:35.326008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:35.326049 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:35.326141 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:35.326564 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.327104 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.327855 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.328061 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.328149 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.330633 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:35799: BlockingRecv error: recv got EOF from 127.13.8.65:35799 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:35.519002 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:35.513611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:45795)
0924 16:07:35.513902 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:35.513916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:35.514015 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:35.514289 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:35.514291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:35.514476 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:35.514595 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.514601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.515340 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.515343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.515937 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.515945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.516376 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.516379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.516446 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.516953 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:35.516965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:35.518883 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
W20260924 16:07:35.519207 22609 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:35.513688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:45795 (local address 127.13.8.126:43869)
0924 16:07:35.513838 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:35.513842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:35.513986 (+ 144us) server_negotiation.cc:407] Connection header received
0924 16:07:35.514082 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:35.514085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:35.514127 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:35.514192 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:35.514712 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.515242 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.516041 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.516267 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.516340 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.519064 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:45795: BlockingRecv error: recv got EOF from 127.13.8.66:45795 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:35 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:35.710942 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:35.704943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:34389)
0924 16:07:35.705263 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:35.705283 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:35.705395 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:35.705658 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:35.705660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:35.705868 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:35.705974 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.705981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.706917 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.706921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.707874 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:35.707905 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.708386 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.708388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:35.708454 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.708950 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:35.708962 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:35.710822 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":115,"threads_started":1}
W20260924 16:07:35.711066 22609 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:35.705036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:34389 (local address 127.13.8.126:43869)
0924 16:07:35.705181 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:35.705184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:35.705340 (+ 156us) server_negotiation.cc:407] Connection header received
0924 16:07:35.705484 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:35.705486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:35.705532 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:35.705593 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:35.706084 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.706785 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.708023 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:35.708281 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:35.708360 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:35.710951 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:34389: BlockingRecv error: recv got EOF from 127.13.8.67:34389 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:36.342408 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:36.331442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:43227)
0924 16:07:36.331788 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:36.331801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:36.331905 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:36.332140 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:36.332142 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:36.332368 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:36.332462 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.332467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.333187 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.333198 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.333895 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.333908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.334452 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.334456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.334556 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.340243 (+ 5687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:36.340256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:36.342205 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":84,"threads_started":1}
W20260924 16:07:36.342609 22613 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:36.331516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:43227 (local address 127.13.8.126:43869)
0924 16:07:36.331816 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:36.331821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:36.331858 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:07:36.331971 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:36.331973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:36.332016 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:36.332121 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:36.332581 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.333100 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.333998 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.334320 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.334445 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.342429 (+ 7984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:43227: BlockingRecv error: recv got EOF from 127.13.8.65:43227 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:36.525622 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:36.519863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:55041)
0924 16:07:36.520210 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:36.520228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:36.520359 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:36.520702 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:36.520706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:36.520951 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:36.521103 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.521111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.522044 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.522049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.522709 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.522719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.523173 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.523177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.523263 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.523724 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:36.523735 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:36.525492 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":122,"threads_started":1}
W20260924 16:07:36.525919 22613 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:36.520056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:55041 (local address 127.13.8.126:43869)
0924 16:07:36.520223 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:36.520228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:36.520301 (+ 73us) server_negotiation.cc:407] Connection header received
0924 16:07:36.520467 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:36.520470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:36.520548 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:36.520633 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:36.521223 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.521936 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.522833 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.523060 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.523171 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.525714 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:55041: BlockingRecv error: recv got EOF from 127.13.8.66:55041 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:36 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:36.717818 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:36.711750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:60659)
0924 16:07:36.712248 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:36.712268 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:36.712464 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:36.712832 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:36.712835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:36.713068 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:36.713170 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.713176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.714002 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.714009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.714640 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:36.714650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.715155 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.715159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:36.715258 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.715779 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:36.715792 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:36.717674 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":81,"threads_started":1}
W20260924 16:07:36.717994 22613 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:36.711873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:60659 (local address 127.13.8.126:43869)
0924 16:07:36.712129 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:36.712134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:36.712377 (+ 243us) server_negotiation.cc:407] Connection header received
0924 16:07:36.712635 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:36.712640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:36.712701 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:36.712806 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:36.713303 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.713877 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.714783 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:36.715031 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:36.715153 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:36.717867 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:60659: BlockingRecv error: recv got EOF from 127.13.8.67:60659 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:37.348747 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:37.343234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:47691)
0924 16:07:37.343514 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:37.343531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:37.343638 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:37.343935 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:37.343940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:37.344185 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:37.344332 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.344343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.345163 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.345166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.345735 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.345744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.346154 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.346156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.346252 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.346726 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:37.346738 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:37.348634 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":93,"threads_started":1}
W20260924 16:07:37.348906 22617 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:37.343304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:47691 (local address 127.13.8.126:43869)
0924 16:07:37.343625 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:37.343629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:37.343642 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:07:37.343727 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:37.343730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:37.343782 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:37.343907 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:37.344461 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.345063 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.345838 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.346052 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.346124 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.348766 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:47691: BlockingRecv error: recv got EOF from 127.13.8.65:47691 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":68,"threads_started":1}
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:37.532572 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:37.526611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:58269)
0924 16:07:37.526930 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:37.526942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:37.527087 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:37.527375 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:37.527378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:37.527609 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:37.527718 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.527724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.528498 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.528504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.529220 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.529231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.529746 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.529749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.529861 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.530522 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:37.530534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:37.532396 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":80,"threads_started":1}
W20260924 16:07:37.532639 22617 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:37.526688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:58269 (local address 127.13.8.126:43869)
0924 16:07:37.526846 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:37.526850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:37.527027 (+ 177us) server_negotiation.cc:407] Connection header received
0924 16:07:37.527204 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:37.527207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:37.527262 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:37.527361 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:37.527845 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.528387 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.529355 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.529627 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.529734 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.532531 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:58269: BlockingRecv error: recv got EOF from 127.13.8.66:58269 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:37 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.67@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:37.724255 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:37.718825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.67:57515)
0924 16:07:37.719133 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:37.719145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:37.719268 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:37.719529 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:37.719532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:37.719713 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:37.719806 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.719811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.720567 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.720570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.721113 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:37.721120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.721529 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.721531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:37.721608 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.722086 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:37.722097 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:37.724081 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":155,"threads_started":1}
W20260924 16:07:37.724319 22617 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:37.718888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:57515 (local address 127.13.8.126:43869)
0924 16:07:37.719089 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:37.719095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:37.719245 (+ 150us) server_negotiation.cc:407] Connection header received
0924 16:07:37.719357 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:37.719359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:37.719411 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:37.719500 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:37.719951 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.720472 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.721219 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:37.721424 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:37.721509 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:37.724211 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.67:57515: BlockingRecv error: recv got EOF from 127.13.8.67:57515 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 24 16:07:38 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:38 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.65@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:38.354753 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:38.349649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.65:45903)
0924 16:07:38.349940 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:38.349954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:38.350039 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:38.350326 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:38.350328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:38.350499 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:38.350615 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:38.350620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.351331 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.351333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:38.351897 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:38.351905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.352305 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.352307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:38.352368 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:38.352827 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:38.352839 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:38.354641 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
W20260924 16:07:38.354947 22621 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:38.349765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:45903 (local address 127.13.8.126:43869)
0924 16:07:38.350004 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:38.350008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:38.350020 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:38.350124 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:38.350127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:38.350169 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:38.350276 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:38.350719 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.351242 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.351994 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.352213 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.352285 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:38.354775 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.65:45903: BlockingRecv error: recv got EOF from 127.13.8.65:45903 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
Sep 24 16:07:38 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
Sep 24 16:07:38 dist-test-slave-vj6n krb5kdc[21550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.8.66@KRBTEST.COM for oryx/127.13.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260924 16:07:38.538710 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:38.533518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43869 (local address 127.13.8.66:46593)
0924 16:07:38.533799 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:38.533813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:38.533902 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:38.534125 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:38.534127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:38.534369 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:38.534468 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:38.534474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.535249 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.535252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:38.535808 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:38.535816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.536252 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.536255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:38.536317 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:38.536813 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:38.536824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:38.538596 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.8.126:43869: Server oryx/127.13.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":113,"threads_started":1}
W20260924 16:07:38.538872 22621 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:38.533603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:46593 (local address 127.13.8.126:43869)
0924 16:07:38.533741 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:38.533745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:38.533876 (+ 131us) server_negotiation.cc:407] Connection header received
0924 16:07:38.533981 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:38.533983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:38.534027 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:38.534092 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:38.534594 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.535143 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.535912 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.536134 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.536217 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:38.538747 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.8.66:46593: BlockingRecv error: recv got EOF from 127.13.8.66:46593 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:07:38.622203 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22139
I20260924 16:07:38.628669 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22292
I20260924 16:07:38.634575 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22427
I20260924 16:07:38.640597 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22068
2026-09-24T16:07:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18055 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T16:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:38Z Disabled control of system clock
I20260924 16:07:38.671553 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35355
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 16:07:38.783150 22626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:38.783404 22626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:38.783437 22626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:38.787461 22626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:38.787521 22626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:38.787549 22626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:38.787569 22626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:38.787585 22626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:38.792163 22626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:38.793233 22626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:38.794643 22626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:38.803886 22634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:38.803881 22631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:38.803948 22626 server_base.cc:1034] running on GCE node
W20260924 16:07:38.803906 22632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:38.804708 22626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:38.805869 22626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:38.807054 22626 hybrid_clock.cc:648] HybridClock initialized: now 1790266058807020 us; error 44 us; skew 500 ppm
I20260924 16:07:38.808911 22626 webserver.cc:460] Webserver started at http://127.13.8.126:37791/ using document root <none> and password file <none>
I20260924 16:07:38.809515 22626 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:38.809618 22626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:38.809865 22626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:38.812034 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "40a69e8a3d35452b8929bab1a45937be"
format_stamp: "Formatted at 2026-09-24 16:07:38 on dist-test-slave-vj6n"
server_key: "5e8bcfcf795275eaa482be5c49e2330a"
server_key_iv: "960ee0ca23dd9578f38d62d4811e18cd"
server_key_version: "encryptionkey@0"
I20260924 16:07:38.812641 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "40a69e8a3d35452b8929bab1a45937be"
format_stamp: "Formatted at 2026-09-24 16:07:38 on dist-test-slave-vj6n"
server_key: "5e8bcfcf795275eaa482be5c49e2330a"
server_key_iv: "960ee0ca23dd9578f38d62d4811e18cd"
server_key_version: "encryptionkey@0"
I20260924 16:07:38.816461 22626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260924 16:07:38.819106 22640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:38.820549 22626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:38.820688 22626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "40a69e8a3d35452b8929bab1a45937be"
format_stamp: "Formatted at 2026-09-24 16:07:38 on dist-test-slave-vj6n"
server_key: "5e8bcfcf795275eaa482be5c49e2330a"
server_key_iv: "960ee0ca23dd9578f38d62d4811e18cd"
server_key_version: "encryptionkey@0"
I20260924 16:07:38.820799 22626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:38.847995 22626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:38.848795 22626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:38.849011 22626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:38.856069 22626 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35355
I20260924 16:07:38.856077 22693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35355 every 8 connection(s)
I20260924 16:07:38.857055 22626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:38.857339 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22626
I20260924 16:07:38.857451 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:38.857801 13345 external_mini_cluster.cc:1428] Setting key 74a1e5e553785fc08ea8947663c81920
I20260924 16:07:38.860646 22694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:38.866303 22694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be: Bootstrap starting.
I20260924 16:07:38.867055 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:38.859206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.13.8.126:35355)
0924 16:07:38.859910 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:38.859930 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:38.859976 (+ 46us) server_negotiation.cc:407] Connection header received
0924 16:07:38.860665 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:38.860682 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:38.861110 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:38.861441 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:38.861874 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.862894 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.863776 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:38.864245 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:38.864422 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:38.865287 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:38.865292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:38.865305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:38.865681 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:38.865735 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:38.865755 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:38.865866 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:38.865981 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 16:07:38.866294 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":143,"threads_started":1}
I20260924 16:07:38.868762 22694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:38.869601 22694 log.cc:824] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:38.871631 22694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be: No bootstrap required, opened a new log
I20260924 16:07:38.875116 22694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } }
I20260924 16:07:38.875327 22694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:38.875361 22694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a69e8a3d35452b8929bab1a45937be, State: Initialized, Role: FOLLOWER
I20260924 16:07:38.875759 22694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } }
I20260924 16:07:38.875877 22694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:38.875921 22694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:38.875993 22694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:38.876973 22694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } }
I20260924 16:07:38.877282 22694 leader_election.cc:302] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40a69e8a3d35452b8929bab1a45937be; no voters:
I20260924 16:07:38.877547 22694 leader_election.cc:288] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:38.877633 22699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:38.877882 22699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [term 1 LEADER]: Becoming Leader. State: Replica: 40a69e8a3d35452b8929bab1a45937be, State: Running, Role: LEADER
I20260924 16:07:38.878266 22699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } }
I20260924 16:07:38.878873 22694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:38.879912 22700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40a69e8a3d35452b8929bab1a45937be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } } }
I20260924 16:07:38.880043 22700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:38.880328 22701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40a69e8a3d35452b8929bab1a45937be. Latest consensus state: current_term: 1 leader_uuid: "40a69e8a3d35452b8929bab1a45937be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a69e8a3d35452b8929bab1a45937be" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35355 } } }
I20260924 16:07:38.880388 22708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:38.880436 22701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:38.883644 22708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:38.890877 22708 catalog_manager.cc:1347] Generated new cluster ID: 8c996375175143d6b4d995a96e1bdaa6
I20260924 16:07:38.890944 22708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:38.906538 22708 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:38.907538 22708 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:38.914168 22708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40a69e8a3d35452b8929bab1a45937be: Generated new TSK 0
I20260924 16:07:38.915019 22708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:38.917759 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--builtin_ntp_servers=127.13.8.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 16:07:39.047639 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:39.047894 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:39.047931 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:39.051945 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:39.051994 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:39.052081 22718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:39.056795 22718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:39.057838 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:39.059207 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:39.068745 22723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.068727 22726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.068728 22724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.068858 22718 server_base.cc:1034] running on GCE node
I20260924 16:07:39.069372 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:39.070057 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:39.071244 22718 hybrid_clock.cc:648] HybridClock initialized: now 1790266059071224 us; error 30 us; skew 500 ppm
I20260924 16:07:39.073138 22718 webserver.cc:460] Webserver started at http://127.13.8.65:34555/ using document root <none> and password file <none>
I20260924 16:07:39.073735 22718 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:39.073824 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:39.074059 22718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:39.076133 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "05c5f85aa5ad465a84cf917b38e181c1"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "c8a1289fb90bf664855c915708ac51ba"
server_key_iv: "7dc46a29d92c8000f18468044a5052ee"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.076710 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "05c5f85aa5ad465a84cf917b38e181c1"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "c8a1289fb90bf664855c915708ac51ba"
server_key_iv: "7dc46a29d92c8000f18468044a5052ee"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.081276 22718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:07:39.084129 22732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.085346 22718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:39.085440 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "05c5f85aa5ad465a84cf917b38e181c1"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "c8a1289fb90bf664855c915708ac51ba"
server_key_iv: "7dc46a29d92c8000f18468044a5052ee"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.085577 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:39.102172 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:39.102929 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:39.103120 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:39.103727 22718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:39.104714 22718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:39.104759 22718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.104828 22718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:39.104859 22718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.113415 22718 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:37253
I20260924 16:07:39.113440 22846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:37253 every 8 connection(s)
I20260924 16:07:39.114410 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:39.114794 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22718
I20260924 16:07:39.114903 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:39.115221 13345 external_mini_cluster.cc:1428] Setting key e28b02b59321dc4eaf76bb7d22867b90
I20260924 16:07:39.117362 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--builtin_ntp_servers=127.13.8.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 16:07:39.125429 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.116389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:40289 (local address 127.13.8.126:35355)
0924 16:07:39.116535 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.116539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.117820 (+ 1281us) server_negotiation.cc:407] Connection header received
0924 16:07:39.119657 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.119660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.119719 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.119805 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.120761 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.121689 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.122906 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.123366 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.123500 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.124760 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:39.124766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:39.124771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:39.124790 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:39.124816 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:39.124830 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:39.124919 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:39.125085 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 16:07:39.125214 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:07:39.126209 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.116880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35355 (local address 127.13.8.65:40289)
0924 16:07:39.117667 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:39.117724 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:39.119431 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:39.120023 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:39.120031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:39.120116 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:39.120612 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.120626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.121830 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.121833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.122786 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.122795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.123559 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.123565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.123783 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.124447 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:39.124451 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:39.124508 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:39.124576 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:39.124925 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:39.124932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:39.124937 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:07:39.125040 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 16:07:39.125337 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":103,"threads_started":1}
I20260924 16:07:39.127538 22847 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35355
I20260924 16:07:39.127815 22847 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:39.128564 22847 heartbeater.cc:511] Master 127.13.8.126:35355 requested a full tablet report, sending...
I20260924 16:07:39.130128 22657 ts_manager.cc:194] Registered new tserver with Master: 05c5f85aa5ad465a84cf917b38e181c1 (127.13.8.65:37253)
I20260924 16:07:39.131376 22657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.65:40289
W20260924 16:07:39.232486 22850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:39.232769 22850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:39.232805 22850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:39.236836 22850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:39.236886 22850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:39.236975 22850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:39.241704 22850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:39.242766 22850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:39.244143 22850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:39.253867 22855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.253906 22850 server_base.cc:1034] running on GCE node
W20260924 16:07:39.253892 22858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.253865 22856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.254679 22850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:39.255337 22850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:39.256556 22850 hybrid_clock.cc:648] HybridClock initialized: now 1790266059256520 us; error 46 us; skew 500 ppm
I20260924 16:07:39.258371 22850 webserver.cc:460] Webserver started at http://127.13.8.66:35757/ using document root <none> and password file <none>
I20260924 16:07:39.258945 22850 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:39.259034 22850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:39.259272 22850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:39.261277 22850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "61c94d9241614c4b8bf5cdaacae420b9"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "ec5da1999f1c53572ddd2173bcc7cb7a"
server_key_iv: "ed73087ffef14627154e7e796801a05a"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.261832 22850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "61c94d9241614c4b8bf5cdaacae420b9"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "ec5da1999f1c53572ddd2173bcc7cb7a"
server_key_iv: "ed73087ffef14627154e7e796801a05a"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.265678 22850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260924 16:07:39.268185 22864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.269415 22850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:39.269559 22850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "61c94d9241614c4b8bf5cdaacae420b9"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "ec5da1999f1c53572ddd2173bcc7cb7a"
server_key_iv: "ed73087ffef14627154e7e796801a05a"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.269677 22850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:39.292258 22850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:39.293027 22850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:39.293241 22850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:39.293859 22850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:39.294863 22850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:39.294934 22850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.295006 22850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:39.295054 22850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.303910 22850 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:45323
I20260924 16:07:39.303947 22978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:45323 every 8 connection(s)
I20260924 16:07:39.304832 22850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:39.313879 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22850
I20260924 16:07:39.314009 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:39.314095 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.306740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:35047 (local address 127.13.8.126:35355)
0924 16:07:39.306885 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.306889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.307912 (+ 1023us) server_negotiation.cc:407] Connection header received
0924 16:07:39.309304 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.309308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.309364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.309477 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.310450 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.310993 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.312134 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.312363 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.312476 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.313574 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:39.313578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:39.313580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:39.313591 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:39.313609 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:39.313616 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:39.313726 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:39.313892 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 16:07:39.313955 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:07:39.314406 13345 external_mini_cluster.cc:1428] Setting key c6778bb3b536797d07f70b5996ede150
I20260924 16:07:39.315028 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.307125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35355 (local address 127.13.8.66:35047)
0924 16:07:39.307771 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:39.307810 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:39.309082 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:39.309654 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:39.309661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:39.309757 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:39.310307 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.310322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.311125 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.311129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.312005 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.312015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.312519 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.312523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.312684 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.313313 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:39.313315 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:39.313353 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:39.313414 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:39.313732 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:39.313743 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:39.313749 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:07:39.313877 (+ 128us) client_negotiation.cc:240] Negotiation successful
0924 16:07:39.314141 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260924 16:07:39.316468 22979 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35355
I20260924 16:07:39.316515 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--builtin_ntp_servers=127.13.8.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 16:07:39.316720 22979 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:39.317313 22979 heartbeater.cc:511] Master 127.13.8.126:35355 requested a full tablet report, sending...
I20260924 16:07:39.318578 22657 ts_manager.cc:194] Registered new tserver with Master: 61c94d9241614c4b8bf5cdaacae420b9 (127.13.8.66:45323)
I20260924 16:07:39.319134 22657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.66:35047
W20260924 16:07:39.428826 22982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:39.429082 22982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:39.429143 22982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:39.433179 22982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:39.433262 22982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:39.433343 22982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:39.438160 22982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:39.439415 22982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:39.440845 22982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:39.450938 22990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.450924 22988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.450922 22987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.451233 22982 server_base.cc:1034] running on GCE node
I20260924 16:07:39.451671 22982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:39.452322 22982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:39.453481 22982 hybrid_clock.cc:648] HybridClock initialized: now 1790266059453456 us; error 37 us; skew 500 ppm
I20260924 16:07:39.455265 22982 webserver.cc:460] Webserver started at http://127.13.8.67:42887/ using document root <none> and password file <none>
I20260924 16:07:39.455842 22982 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:39.455914 22982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:39.456182 22982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:39.458334 22982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "c45c533d717e493f918016194e15560a"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "3518c211bf01eaf83ecf8a6d3348046c"
server_key_iv: "29c0949d6d26a7324bf38755f839c575"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.458938 22982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "c45c533d717e493f918016194e15560a"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "3518c211bf01eaf83ecf8a6d3348046c"
server_key_iv: "29c0949d6d26a7324bf38755f839c575"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.462556 22982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260924 16:07:39.464939 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.466178 22982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:39.466373 22982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "c45c533d717e493f918016194e15560a"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "3518c211bf01eaf83ecf8a6d3348046c"
server_key_iv: "29c0949d6d26a7324bf38755f839c575"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.466476 22982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:39.493696 22982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:39.494503 22982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:39.494693 22982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:39.495316 22982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:39.496294 22982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:39.496340 22982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.496407 22982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:39.496445 22982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.505075 22982 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:34747
I20260924 16:07:39.505115 23110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:34747 every 8 connection(s)
I20260924 16:07:39.505991 22982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:39.513121 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 22982
I20260924 16:07:39.513201 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:39.513478 13345 external_mini_cluster.cc:1428] Setting key 1f32e83b952bc0d214e5a04719622e46
I20260924 16:07:39.515621 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.507889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:49429 (local address 127.13.8.126:35355)
0924 16:07:39.508021 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.508028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.509104 (+ 1076us) server_negotiation.cc:407] Connection header received
0924 16:07:39.510592 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.510595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.510653 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.510771 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.511705 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.512231 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.513472 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.513769 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.513878 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.515129 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:39.515132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:39.515134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:39.515144 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:39.515160 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:39.515168 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:39.515229 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:39.515394 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 16:07:39.515439 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:39.516458 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.508295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35355 (local address 127.13.8.67:49429)
0924 16:07:39.508959 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:39.508999 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:39.510375 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:39.510963 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:39.510970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:39.511056 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:39.511553 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.511566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.512388 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.512391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.513339 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:39.513349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:39.513923 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:39.513927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:39.514088 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:39.514835 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:39.514837 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:39.514903 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:39.514987 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:39.515252 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:39.515260 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:39.515266 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:07:39.515403 (+ 137us) client_negotiation.cc:240] Negotiation successful
0924 16:07:39.515689 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260924 16:07:39.517590 23111 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35355
I20260924 16:07:39.517803 23111 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:39.518289 23111 heartbeater.cc:511] Master 127.13.8.126:35355 requested a full tablet report, sending...
I20260924 16:07:39.519219 22657 ts_manager.cc:194] Registered new tserver with Master: c45c533d717e493f918016194e15560a (127.13.8.67:34747)
I20260924 16:07:39.519732 22657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.67:49429
I20260924 16:07:39.527545 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 16:07:39.530418 22697 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:39.529456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.13.8.126:35355)
0924 16:07:39.529720 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.529724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.529739 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:07:39.529837 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.529840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.529883 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.529960 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.530203 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48214: BlockingRecv error: recv got EOF from 127.0.0.1:48214 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
I20260924 16:07:39.530462 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22718
I20260924 16:07:39.537823 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22850
I20260924 16:07:39.543617 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22982
I20260924 16:07:39.549180 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 22626
2026-09-24T16:07:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (907 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T16:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:39Z Disabled control of system clock
I20260924 16:07:39.579658 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:32829
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:32829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 16:07:39.692633 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:39.692881 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:39.692935 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:39.696903 23123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:39.696955 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:39.696970 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:39.696986 23123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:39.697001 23123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:39.701700 23123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:32829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:39.702730 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:39.704061 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:39.712484 23131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.712561 23128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.712495 23129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.712757 23123 server_base.cc:1034] running on GCE node
I20260924 16:07:39.713352 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:39.714501 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:39.715716 23123 hybrid_clock.cc:648] HybridClock initialized: now 1790266059715697 us; error 35 us; skew 500 ppm
I20260924 16:07:39.717720 23123 webserver.cc:460] Webserver started at http://127.13.8.126:38469/ using document root <none> and password file <none>
I20260924 16:07:39.718412 23123 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:39.718519 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:39.718766 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:39.721166 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "e0005664ae8b4a7b8e90f9d2d73f4500"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "32c4c6408f250b06eedb53b8b4eab38e"
server_key_iv: "055f3eaf868b741e12f5e523c3498947"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.721830 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "e0005664ae8b4a7b8e90f9d2d73f4500"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "32c4c6408f250b06eedb53b8b4eab38e"
server_key_iv: "055f3eaf868b741e12f5e523c3498947"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.725827 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260924 16:07:39.728370 23137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.729557 23123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:39.729704 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "e0005664ae8b4a7b8e90f9d2d73f4500"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "32c4c6408f250b06eedb53b8b4eab38e"
server_key_iv: "055f3eaf868b741e12f5e523c3498947"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.729823 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:39.734943 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:39.735610 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:39.735805 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:39.743058 23123 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:32829
I20260924 16:07:39.743068 23189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:32829 every 8 connection(s)
I20260924 16:07:39.744020 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:39.745510 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23123
I20260924 16:07:39.745626 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:39.745929 13345 external_mini_cluster.cc:1428] Setting key 18eeec6aa50f212cc4f179929ec099a4
I20260924 16:07:39.747922 23191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:39.752029 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.747589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.13.8.126:32829)
0924 16:07:39.748299 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.748316 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.748354 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:07:39.749039 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.749055 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.749421 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.749815 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.750675 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:39.750679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:39.750691 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:39.750700 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:39.750755 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:39.750779 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:39.750895 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:39.751013 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 16:07:39.751286 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
I20260924 16:07:39.754014 23191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: Bootstrap starting.
I20260924 16:07:39.756152 23191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:39.756896 23191 log.cc:824] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:39.758849 23191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: No bootstrap required, opened a new log
I20260924 16:07:39.762283 23191 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } }
I20260924 16:07:39.762493 23191 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:39.762528 23191 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0005664ae8b4a7b8e90f9d2d73f4500, State: Initialized, Role: FOLLOWER
I20260924 16:07:39.762951 23191 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } }
I20260924 16:07:39.763067 23191 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:39.763150 23191 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:39.763267 23191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:39.764102 23191 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } }
I20260924 16:07:39.764513 23191 leader_election.cc:302] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0005664ae8b4a7b8e90f9d2d73f4500; no voters:
I20260924 16:07:39.764827 23191 leader_election.cc:288] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:39.764955 23196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:39.765251 23196 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [term 1 LEADER]: Becoming Leader. State: Replica: e0005664ae8b4a7b8e90f9d2d73f4500, State: Running, Role: LEADER
I20260924 16:07:39.765576 23196 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } }
I20260924 16:07:39.766048 23191 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:39.767350 23198 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0005664ae8b4a7b8e90f9d2d73f4500. Latest consensus state: current_term: 1 leader_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } } }
I20260924 16:07:39.767361 23197 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0005664ae8b4a7b8e90f9d2d73f4500" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32829 } } }
I20260924 16:07:39.767539 23197 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:39.767776 23198 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500 [sys.catalog]: This master's current role is: LEADER
W20260924 16:07:39.771346 23211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:07:39.771443 23211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:07:39.771522 23207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:39.772301 23207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:39.777467 23207 catalog_manager.cc:1347] Generated new cluster ID: 502a6e9a8e644a048d48ffebe35faf4b
I20260924 16:07:39.777529 23207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:39.793255 23207 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:39.793617 23207 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:39.801946 23207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0005664ae8b4a7b8e90f9d2d73f4500: Generated new TSK 0
I20260924 16:07:39.802804 23207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:07:39.805449 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--builtin_ntp_servers=127.13.8.84:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 16:07:39.917670 23215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:39.918001 23215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:39.918082 23215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:39.922125 23215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:39.922233 23215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:39.922329 23215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:39.927353 23215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:39.928498 23215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:39.929899 23215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:39.940186 23220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:39.940176 23221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.940245 23215 server_base.cc:1034] running on GCE node
W20260924 16:07:39.940176 23223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:39.940793 23215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:39.941438 23215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:39.942619 23215 hybrid_clock.cc:648] HybridClock initialized: now 1790266059942584 us; error 44 us; skew 500 ppm
I20260924 16:07:39.944554 23215 webserver.cc:460] Webserver started at http://127.13.8.65:46007/ using document root <none> and password file <none>
I20260924 16:07:39.945154 23215 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:39.945250 23215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:39.945484 23215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:39.947696 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "5d3fffde948e41bdabe3623470ff1dbc"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "8113ed04f025daf593601a7653566834"
server_key_iv: "9099d3ef168629cefc5396b6d735c11f"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.948288 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "5d3fffde948e41bdabe3623470ff1dbc"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "8113ed04f025daf593601a7653566834"
server_key_iv: "9099d3ef168629cefc5396b6d735c11f"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.952105 23215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:07:39.954680 23229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.955904 23215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:39.956038 23215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "5d3fffde948e41bdabe3623470ff1dbc"
format_stamp: "Formatted at 2026-09-24 16:07:39 on dist-test-slave-vj6n"
server_key: "8113ed04f025daf593601a7653566834"
server_key_iv: "9099d3ef168629cefc5396b6d735c11f"
server_key_version: "encryptionkey@0"
I20260924 16:07:39.956153 23215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:39.960681 23215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:39.961284 23215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:39.961470 23215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:39.961957 23215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:39.962877 23215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:39.962922 23215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.962960 23215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:39.962975 23215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:39.971743 23215 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:33697
I20260924 16:07:39.971761 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:33697 every 8 connection(s)
I20260924 16:07:39.972620 23215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:39.978780 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.974453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:40599 (local address 127.13.8.126:32829)
0924 16:07:39.974605 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:39.974608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:39.975703 (+ 1095us) server_negotiation.cc:407] Connection header received
0924 16:07:39.977030 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:39.977034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:39.977079 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:39.977164 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:39.978304 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:39.978307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:39.978309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:39.978321 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:39.978341 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:39.978357 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:39.978455 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:39.978593 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 16:07:39.978639 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:07:39.979600 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:39.974886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32829 (local address 127.13.8.65:40599)
0924 16:07:39.975546 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:39.975585 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:39.976813 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:39.977341 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:39.977349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:39.977436 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:39.978021 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:39.978022 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:39.978062 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:39.978130 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:39.978439 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:39.978447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:39.978453 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 16:07:39.978639 (+ 186us) client_negotiation.cc:240] Negotiation successful
0924 16:07:39.978895 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260924 16:07:39.980850 23344 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32829
I20260924 16:07:39.981097 23344 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:39.981515 23344 heartbeater.cc:511] Master 127.13.8.126:32829 requested a full tablet report, sending...
I20260924 16:07:39.981853 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23215
I20260924 16:07:39.981938 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:39.982292 13345 external_mini_cluster.cc:1428] Setting key ab39c72eda0ff0dfb94a305c797c421e
I20260924 16:07:39.983281 23154 ts_manager.cc:194] Registered new tserver with Master: 5d3fffde948e41bdabe3623470ff1dbc (127.13.8.65:33697)
I20260924 16:07:39.984496 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--builtin_ntp_servers=127.13.8.84:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 16:07:40.101286 23347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:40.101568 23347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:40.101641 23347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:40.105793 23347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:40.105855 23347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:40.105958 23347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:40.111047 23347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:40.112186 23347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:40.113620 23347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:40.123503 23353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:40.123638 23347 server_base.cc:1034] running on GCE node
W20260924 16:07:40.123549 23352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:40.123827 23355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:40.124226 23347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:40.124866 23347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:40.126034 23347 hybrid_clock.cc:648] HybridClock initialized: now 1790266060126018 us; error 25 us; skew 500 ppm
I20260924 16:07:40.127866 23347 webserver.cc:460] Webserver started at http://127.13.8.66:41165/ using document root <none> and password file <none>
I20260924 16:07:40.128466 23347 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:40.128526 23347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:40.128779 23347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:40.130915 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "7a40f94fb0214cbb8fa75298990b8c6c"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "c97c6c7594d6123f2dd7a2935134e517"
server_key_iv: "70b67c042b284123146be31ddfa95e84"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.131492 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "7a40f94fb0214cbb8fa75298990b8c6c"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "c97c6c7594d6123f2dd7a2935134e517"
server_key_iv: "70b67c042b284123146be31ddfa95e84"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.135318 23347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:07:40.137902 23361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.139138 23347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:40.139276 23347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "7a40f94fb0214cbb8fa75298990b8c6c"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "c97c6c7594d6123f2dd7a2935134e517"
server_key_iv: "70b67c042b284123146be31ddfa95e84"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.139391 23347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:40.143922 23347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:40.144548 23347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:40.144773 23347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:40.145323 23347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:40.146313 23347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:40.146379 23347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.146445 23347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:40.146497 23347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.155555 23347 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:44901
I20260924 16:07:40.155573 23475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:44901 every 8 connection(s)
I20260924 16:07:40.156560 23347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:40.161155 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23347
I20260924 16:07:40.161316 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:40.161655 13345 external_mini_cluster.cc:1428] Setting key e356465fbefc381507fd88b97b1ecf3d
I20260924 16:07:40.163916 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:40.158587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:48729 (local address 127.13.8.126:32829)
0924 16:07:40.158737 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:40.158741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:40.160176 (+ 1435us) server_negotiation.cc:407] Connection header received
0924 16:07:40.161890 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:40.161894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:40.161943 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:40.162025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:40.163476 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:40.163479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:40.163481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:40.163493 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:40.163514 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:40.163522 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:40.163595 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:40.163781 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 16:07:40.163816 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 16:07:40.164016 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--builtin_ntp_servers=127.13.8.84:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260924 16:07:40.165292 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:40.159126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32829 (local address 127.13.8.66:48729)
0924 16:07:40.159958 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:40.160024 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:40.161597 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:40.162313 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:40.162322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:40.162462 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:40.163147 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:40.163149 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:40.163197 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:40.163314 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:40.163613 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:40.163623 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:40.163631 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 16:07:40.163850 (+ 219us) client_negotiation.cc:240] Negotiation successful
0924 16:07:40.164147 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":128,"threads_started":1}
I20260924 16:07:40.166649 23476 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32829
I20260924 16:07:40.166944 23476 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:40.167389 23476 heartbeater.cc:511] Master 127.13.8.126:32829 requested a full tablet report, sending...
I20260924 16:07:40.168566 23154 ts_manager.cc:194] Registered new tserver with Master: 7a40f94fb0214cbb8fa75298990b8c6c (127.13.8.66:44901)
W20260924 16:07:40.280131 23479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:40.280623 23479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:40.280743 23479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:40.284970 23479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:40.285058 23479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:40.285142 23479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:40.290037 23479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:40.291230 23479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:40.292654 23479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:40.302270 23485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:40.302270 23487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:40.302316 23484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:40.302465 23479 server_base.cc:1034] running on GCE node
I20260924 16:07:40.303007 23479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:40.303689 23479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:40.304873 23479 hybrid_clock.cc:648] HybridClock initialized: now 1790266060304854 us; error 35 us; skew 500 ppm
I20260924 16:07:40.306842 23479 webserver.cc:460] Webserver started at http://127.13.8.67:35021/ using document root <none> and password file <none>
I20260924 16:07:40.307407 23479 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:40.307466 23479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:40.307638 23479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:40.309630 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "b1cce27b73174da0b856921d02448d71"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "fa3bf93224b7a65634eaac27705bc4b8"
server_key_iv: "de8701cd7ccc6ee537274b130be19f64"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.310122 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "b1cce27b73174da0b856921d02448d71"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "fa3bf93224b7a65634eaac27705bc4b8"
server_key_iv: "de8701cd7ccc6ee537274b130be19f64"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.314054 23479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260924 16:07:40.316649 23493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.317939 23479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:07:40.318086 23479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "b1cce27b73174da0b856921d02448d71"
format_stamp: "Formatted at 2026-09-24 16:07:40 on dist-test-slave-vj6n"
server_key: "fa3bf93224b7a65634eaac27705bc4b8"
server_key_iv: "de8701cd7ccc6ee537274b130be19f64"
server_key_version: "encryptionkey@0"
I20260924 16:07:40.318197 23479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:40.323033 23479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:40.323661 23479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:40.323889 23479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:40.324429 23479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:40.325346 23479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:40.325389 23479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.325451 23479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:40.325484 23479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:40.335219 23479 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:33183
I20260924 16:07:40.335263 23607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:33183 every 8 connection(s)
I20260924 16:07:40.336184 23479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:40.340986 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23479
I20260924 16:07:40.341104 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:40.341429 13345 external_mini_cluster.cc:1428] Setting key d011d3180e9d8c7c1ec0860d5a71ee92
I20260924 16:07:40.343044 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:40.338324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:59317 (local address 127.13.8.126:32829)
0924 16:07:40.338456 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:40.338459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:40.339590 (+ 1131us) server_negotiation.cc:407] Connection header received
0924 16:07:40.340978 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:40.340982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:40.341025 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:40.341177 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:40.342470 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:40.342473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:40.342476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:07:40.342496 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:40.342522 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:07:40.342536 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:40.342666 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:40.342855 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 16:07:40.342916 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 16:07:40.344117 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:40.338702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32829 (local address 127.13.8.67:59317)
0924 16:07:40.339397 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:40.339436 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:40.340684 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:40.341399 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:40.341409 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:40.341527 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:40.342144 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:07:40.342145 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:40.342188 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:07:40.342313 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:40.342660 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:40.342672 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:40.342679 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 16:07:40.342896 (+ 217us) client_negotiation.cc:240] Negotiation successful
0924 16:07:40.343189 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":128,"threads_started":1}
I20260924 16:07:40.345124 23608 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32829
I20260924 16:07:40.345347 23608 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:40.345769 23608 heartbeater.cc:511] Master 127.13.8.126:32829 requested a full tablet report, sending...
I20260924 16:07:40.346822 23154 ts_manager.cc:194] Registered new tserver with Master: b1cce27b73174da0b856921d02448d71 (127.13.8.67:33183)
I20260924 16:07:40.355731 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 16:07:40.358479 23194 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:07:40.357639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.13.8.126:32829)
0924 16:07:40.357768 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:40.357777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:40.357926 (+ 149us) server_negotiation.cc:407] Connection header received
0924 16:07:40.358015 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:40.358017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:40.358048 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:40.358112 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:40.358329 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49112: BlockingRecv error: recv got EOF from 127.0.0.1:49112 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:07:40.358922 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23215
I20260924 16:07:40.365298 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23347
I20260924 16:07:40.371186 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23479
I20260924 16:07:40.376770 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23123
2026-09-24T16:07:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (827 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:40 dist-test-slave-vj6n krb5kdc[23618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:40 dist-test-slave-vj6n krb5kdc[23618](info): set up 2 sockets
Sep 24 16:07:40 dist-test-slave-vj6n krb5kdc[23618](info): commencing operation
krb5kdc: starting...
W20260924 16:07:41.412220 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:41 dist-test-slave-vj6n krb5kdc[23618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:07:41.534536 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36659
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:41.645134 23631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:41.645370 23631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:41.645407 23631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:41.649327 23631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:07:41.649381 23631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:41.649439 23631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:41.649477 23631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:07:41.649510 23631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:07:41.654944 23631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:41.655987 23631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:41.657020 23631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:41.665247 23639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:41.665256 23637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:41.665314 23636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:41.665756 23631 server_base.cc:1034] running on GCE node
I20260924 16:07:41.666326 23631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:41.667426 23631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:41.668613 23631 hybrid_clock.cc:648] HybridClock initialized: now 1790266061668601 us; error 24 us; skew 500 ppm
Sep 24 16:07:41 dist-test-slave-vj6n krb5kdc[23618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:41.676548 23631 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:07:41.677359 23631 webserver.cc:460] Webserver started at http://127.13.8.126:37071/ using document root <none> and password file <none>
I20260924 16:07:41.677871 23631 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:41.677927 23631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:41.678085 23631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:41.680145 23631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "35790270da4a43c4b8a903021102c6e9"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "97a555140cf90473880d815e1a992198"
server_key_iv: "ac98899b47c5907aed168cdf5f754e7f"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.680646 23631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "35790270da4a43c4b8a903021102c6e9"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "97a555140cf90473880d815e1a992198"
server_key_iv: "ac98899b47c5907aed168cdf5f754e7f"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.684175 23631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260924 16:07:41.686496 23646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:41.687691 23631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:07:41.687826 23631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "35790270da4a43c4b8a903021102c6e9"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "97a555140cf90473880d815e1a992198"
server_key_iv: "ac98899b47c5907aed168cdf5f754e7f"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.687952 23631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:41.701112 23631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:41.703895 23631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:41.704061 23631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:41.710434 23631 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:36659
I20260924 16:07:41.710738 23698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:36659 every 8 connection(s)
I20260924 16:07:41.711467 23631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:07:41.714118 23700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:41.719467 23700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: Bootstrap starting.
I20260924 16:07:41.720721 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23631
I20260924 16:07:41.720845 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:07:41.721136 13345 external_mini_cluster.cc:1428] Setting key bd8f7f3e26d32e59a227ab7430b30bb2
I20260924 16:07:41.722302 23700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:41.723096 23700 log.cc:824] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:41.724908 23700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: No bootstrap required, opened a new log
I20260924 16:07:41.728426 23700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } }
I20260924 16:07:41.728674 23700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:41.728753 23700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35790270da4a43c4b8a903021102c6e9, State: Initialized, Role: FOLLOWER
I20260924 16:07:41.729228 23700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } }
I20260924 16:07:41.729384 23700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:07:41.729460 23700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:07:41.729590 23700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 16:07:41 dist-test-slave-vj6n krb5kdc[23618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:41.730572 23700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } }
I20260924 16:07:41.730957 23700 leader_election.cc:302] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35790270da4a43c4b8a903021102c6e9; no voters:
I20260924 16:07:41.731276 23700 leader_election.cc:288] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:07:41.731369 23705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:41.731592 23705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [term 1 LEADER]: Becoming Leader. State: Replica: 35790270da4a43c4b8a903021102c6e9, State: Running, Role: LEADER
I20260924 16:07:41.731935 23705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } }
I20260924 16:07:41.732663 23700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:07:41.733600 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35790270da4a43c4b8a903021102c6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } } }
I20260924 16:07:41.733726 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:41.734107 23707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35790270da4a43c4b8a903021102c6e9. Latest consensus state: current_term: 1 leader_uuid: "35790270da4a43c4b8a903021102c6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35790270da4a43c4b8a903021102c6e9" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36659 } } }
I20260924 16:07:41.734282 23707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9 [sys.catalog]: This master's current role is: LEADER
I20260924 16:07:41.734588 23714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:07:41.736124 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:41.722304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.13.8.126:36659)
0924 16:07:41.722723 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:41.722738 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:41.722771 (+ 33us) server_negotiation.cc:407] Connection header received
0924 16:07:41.723450 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:41.723469 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:41.723771 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:41.724119 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:41.724911 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:41.725921 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:41.726855 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:41.727332 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:41.727508 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:41.730337 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:41.730343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:41.730356 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:41.730383 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:41.732396 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:41.732893 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:41.732899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:41.732906 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:41.733070 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:41.733412 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:41.733418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:41.733423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:41.733991 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:41.734861 (+ 870us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:41.735029 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 16:07:41.735357 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":127,"threads_started":1}
I20260924 16:07:41.738417 23714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:07:41.743790 23714 catalog_manager.cc:1347] Generated new cluster ID: 714a9aa901ba46cba27a60a35f6f5203
I20260924 16:07:41.743855 23714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:07:41.756589 23714 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:07:41.757844 23714 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:07:41.769232 23714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: Generated new TSK 0
I20260924 16:07:41.770102 23714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:07:41.800391 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:41.933171 23726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:41.933454 23726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:41.933542 23726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:41.937636 23726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:41.937702 23726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:41.937762 23726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:07:41.943192 23726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:41.944324 23726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:41.945333 23726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:41.954905 23731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:41.954902 23734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:41.954917 23732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:41.955219 23726 server_base.cc:1034] running on GCE node
I20260924 16:07:41.956176 23726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:41.956842 23726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:41.958043 23726 hybrid_clock.cc:648] HybridClock initialized: now 1790266061958011 us; error 48 us; skew 500 ppm
Sep 24 16:07:41 dist-test-slave-vj6n krb5kdc[23618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:41.960814 23726 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:07:41.961665 23726 webserver.cc:460] Webserver started at http://127.13.8.65:44531/ using document root <none> and password file <none>
I20260924 16:07:41.962307 23726 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:41.962400 23726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:41.962644 23726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:41.964649 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "8aa94b9adca4410ca3451eda6af035b5"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "6174016a2e82eeb4c79f5d7078eac717"
server_key_iv: "ed763451c7cb060fc30013f784238d2c"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.965198 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "8aa94b9adca4410ca3451eda6af035b5"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "6174016a2e82eeb4c79f5d7078eac717"
server_key_iv: "ed763451c7cb060fc30013f784238d2c"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.969058 23726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:07:41.971483 23741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:41.972653 23726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:07:41.972783 23726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "8aa94b9adca4410ca3451eda6af035b5"
format_stamp: "Formatted at 2026-09-24 16:07:41 on dist-test-slave-vj6n"
server_key: "6174016a2e82eeb4c79f5d7078eac717"
server_key_iv: "ed763451c7cb060fc30013f784238d2c"
server_key_version: "encryptionkey@0"
I20260924 16:07:41.972890 23726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:41.986672 23726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:41.989348 23726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:41.989535 23726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:41.990123 23726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:41.991132 23726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:41.991206 23726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:41.991289 23726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:41.991340 23726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.000380 23726 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:44205
I20260924 16:07:42.000398 23854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:44205 every 8 connection(s)
I20260924 16:07:42.001317 23726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:07:42.006937 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23726
I20260924 16:07:42.007062 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:07:42.007411 13345 external_mini_cluster.cc:1428] Setting key 4b5e2b4004a8c49eedb5775a52c0ed3d
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[23618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266061, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:42.016650 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.003180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:49465 (local address 127.13.8.126:36659)
0924 16:07:42.003325 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.003329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.004301 (+ 972us) server_negotiation.cc:407] Connection header received
0924 16:07:42.005718 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.005722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.005784 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.005895 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:42.007372 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.008074 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.009597 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.009816 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.009895 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.012745 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:42.012748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:42.012750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:42.012773 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:42.014349 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.015007 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.015009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.015010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.015095 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.015486 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.015488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.015489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.015653 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:42.015745 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.016423 (+ 678us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.016525 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:07:42.018005 23857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.003659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36659 (local address 127.13.8.65:49465)
0924 16:07:42.004135 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.004178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.005485 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.006077 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.006084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.006476 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:42.007167 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.007207 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.008202 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.008212 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.009467 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.009482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.009972 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.009978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.010239 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.010994 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:42.011011 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:42.012570 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:42.014460 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.014469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.014490 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.014905 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.015187 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.015191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.015196 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.015392 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.015780 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:42.015788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:42.016131 (+ 343us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.016393 (+ 262us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.016788 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
I20260924 16:07:42.019327 23855 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36659
I20260924 16:07:42.019584 23855 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:42.020162 23855 heartbeater.cc:511] Master 127.13.8.126:36659 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:07:42.021759 23663 ts_manager.cc:194] Registered new tserver with Master: 8aa94b9adca4410ca3451eda6af035b5 (127.13.8.65:44205)
I20260924 16:07:42.023404 23663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:49465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:07:42.037273 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:42.152697 23860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:42.152922 23860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:42.152966 23860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:42.157004 23860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:42.157059 23860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:42.157116 23860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:07:42.162964 23860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:42.164012 23860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:42.165038 23860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:42.175115 23866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:42.175113 23867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:42.175310 23860 server_base.cc:1034] running on GCE node
W20260924 16:07:42.175113 23869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:42.176103 23860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:42.177103 23860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:42.178313 23860 hybrid_clock.cc:648] HybridClock initialized: now 1790266062178267 us; error 51 us; skew 500 ppm
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[23618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:42.181913 23860 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:07:42.182928 23860 webserver.cc:460] Webserver started at http://127.13.8.66:39645/ using document root <none> and password file <none>
I20260924 16:07:42.183537 23860 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:42.183629 23860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:42.183893 23860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:42.186024 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "4252f61b4cb940c393fc5e49a05309cf"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "43fde9336cd4e87d233deed46f0af391"
server_key_iv: "dc67e93e2415ed5eef89592f86d3cf12"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.186690 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "4252f61b4cb940c393fc5e49a05309cf"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "43fde9336cd4e87d233deed46f0af391"
server_key_iv: "dc67e93e2415ed5eef89592f86d3cf12"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.190888 23860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:07:42.193739 23876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.195128 23860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:42.195273 23860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "4252f61b4cb940c393fc5e49a05309cf"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "43fde9336cd4e87d233deed46f0af391"
server_key_iv: "dc67e93e2415ed5eef89592f86d3cf12"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.195391 23860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:42.210376 23860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:42.213717 23860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:42.213919 23860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:42.214591 23860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:42.215415 23860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:42.215482 23860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.215550 23860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:42.215600 23860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.225986 23860 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:42575
I20260924 16:07:42.226044 23991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:42575 every 8 connection(s)
I20260924 16:07:42.226974 23860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:07:42.233386 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23860
I20260924 16:07:42.233506 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:07:42.233841 13345 external_mini_cluster.cc:1428] Setting key 69d7c31946fec2570917c4fe4520d9bb
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[23618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:07:42.252796 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.229421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:50005 (local address 127.13.8.126:36659)
0924 16:07:42.229576 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.229581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.230660 (+ 1079us) server_negotiation.cc:407] Connection header received
0924 16:07:42.232874 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.232877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.232928 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.233004 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:42.234907 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.235836 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.237911 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.238135 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.238284 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.248525 (+ 10241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:42.248531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:42.248534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:42.248572 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:42.250409 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.251044 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.251047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.251049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.251138 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.251514 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.251520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.251525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.251755 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:42.251866 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.252495 (+ 629us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.252626 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:07:42.253927 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.229873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36659 (local address 127.13.8.66:50005)
0924 16:07:42.230468 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.230532 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.232573 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.233306 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.233319 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.233870 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:42.234716 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.234739 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.236023 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.236029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.237759 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.237775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.238337 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.238343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.238601 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.244274 (+ 5673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:42.244309 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:42.248261 (+ 3952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:42.250599 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.250607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.250629 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.250922 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.251242 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.251245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.251247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.251405 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.251868 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:42.251878 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:42.252237 (+ 359us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.252441 (+ 204us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.252824 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":99,"threads_started":1}
I20260924 16:07:42.255461 23992 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36659
I20260924 16:07:42.255752 23992 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:42.256439 23992 heartbeater.cc:511] Master 127.13.8.126:36659 requested a full tablet report, sending...
I20260924 16:07:42.257633 23663 ts_manager.cc:194] Registered new tserver with Master: 4252f61b4cb940c393fc5e49a05309cf (127.13.8.66:42575)
I20260924 16:07:42.258296 23663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:50005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:07:42.273161 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:07:42.383553 23997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:07:42.383801 23997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:07:42.383843 23997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:07:42.387880 23997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:07:42.387945 23997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:07:42.388003 23997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:07:42.393487 23997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:07:42.394605 23997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:07:42.395668 23997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:07:42.405442 24002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:07:42.405512 24003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:42.405565 23997 server_base.cc:1034] running on GCE node
W20260924 16:07:42.405512 24005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:07:42.406137 23997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:07:42.406805 23997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:07:42.407989 23997 hybrid_clock.cc:648] HybridClock initialized: now 1790266062407972 us; error 26 us; skew 500 ppm
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[23618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:07:42.410607 23997 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:07:42.411463 23997 webserver.cc:460] Webserver started at http://127.13.8.67:45893/ using document root <none> and password file <none>
I20260924 16:07:42.412040 23997 fs_manager.cc:359] Metadata directory not provided
I20260924 16:07:42.412127 23997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:07:42.412360 23997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:07:42.414444 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "1d9f729505de4e07a840d0cbc64616ec"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "e69a3b577b6467fd079082ffb942e451"
server_key_iv: "80e0745cb665c5d624e04842534a5e6a"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.415016 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "1d9f729505de4e07a840d0cbc64616ec"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "e69a3b577b6467fd079082ffb942e451"
server_key_iv: "80e0745cb665c5d624e04842534a5e6a"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.418776 23997 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:07:42.421186 24012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.422410 23997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:07:42.422535 23997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "1d9f729505de4e07a840d0cbc64616ec"
format_stamp: "Formatted at 2026-09-24 16:07:42 on dist-test-slave-vj6n"
server_key: "e69a3b577b6467fd079082ffb942e451"
server_key_iv: "80e0745cb665c5d624e04842534a5e6a"
server_key_version: "encryptionkey@0"
I20260924 16:07:42.422638 23997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:07:42.448849 23997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:07:42.452299 23997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:07:42.452555 23997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:07:42.453171 23997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:07:42.454026 23997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:07:42.454072 23997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.454135 23997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:07:42.454170 23997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:07:42.463021 23997 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:44789
I20260924 16:07:42.463044 24126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:44789 every 8 connection(s)
I20260924 16:07:42.463912 23997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:07:42.469627 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 23997
I20260924 16:07:42.469771 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:07:42.470046 13345 external_mini_cluster.cc:1428] Setting key ccb0117d514e4dd72dbaa8d59368ce7b
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[23618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:07:42.477396 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.465807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:56533 (local address 127.13.8.126:36659)
0924 16:07:42.465960 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.465964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.467126 (+ 1162us) server_negotiation.cc:407] Connection header received
0924 16:07:42.468051 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.468055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.468113 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.468219 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:42.469366 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.469975 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.471590 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.471805 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.471890 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.474291 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:42.474294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:42.474296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:42.474320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:42.475458 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.475904 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.475906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.475907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.475975 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.476292 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.476294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.476296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.476540 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:42.476645 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.477128 (+ 483us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.477245 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 16:07:42.478313 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.466198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36659 (local address 127.13.8.67:56533)
0924 16:07:42.466978 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.467020 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.467846 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.468397 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.468404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.468744 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:07:42.469209 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.469222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.470094 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.470102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.471453 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.471463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.471953 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.471957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.472166 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.472741 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:07:42.472757 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:07:42.474106 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:07:42.475557 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.475571 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.475582 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.475822 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.476053 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:07:42.476055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:07:42.476056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:07:42.476182 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:07:42.476657 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:07:42.476665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:07:42.476898 (+ 233us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.477087 (+ 189us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.477421 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":112,"threads_started":1}
I20260924 16:07:42.479306 24127 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36659
I20260924 16:07:42.479542 24127 heartbeater.cc:464] Registering TS with master...
I20260924 16:07:42.480010 24127 heartbeater.cc:511] Master 127.13.8.126:36659 requested a full tablet report, sending...
I20260924 16:07:42.480995 23663 ts_manager.cc:194] Registered new tserver with Master: 1d9f729505de4e07a840d0cbc64616ec (127.13.8.67:44789)
I20260924 16:07:42.481541 23663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:56533
I20260924 16:07:42.483830 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:07:42.492372 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.485655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.13.8.126:36659)
0924 16:07:42.485773 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.485776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.485910 (+ 134us) server_negotiation.cc:407] Connection header received
0924 16:07:42.485990 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.485993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.486026 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.486085 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:07:42.486759 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.487222 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.487979 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.488158 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.488225 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.489500 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:07:42.489502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:07:42.489504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:07:42.489520 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:07:42.490803 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.491239 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.491241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.491243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.491312 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:07:42.491529 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:07:42.491531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:07:42.491532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:07:42.491776 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:07:42.491905 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.492175 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.492261 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:07:42.495316 23663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:07:42.498337 23663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:07:42.513020 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.508244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.13.8.66:42575)
0924 16:07:42.508904 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.508914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.508931 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:07:42.508986 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.508989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.509111 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.509263 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:42.509478 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.510361 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.511776 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512333 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512444 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.512585 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.512767 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.512860 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":89,"threads_started":1}
I20260924 16:07:42.513053 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.507203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44205 (local address 127.0.0.1:56042)
0924 16:07:42.507637 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.507650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.507732 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.508644 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.508647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.508660 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:42.508767 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.508773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.510058 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.510062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.511439 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.511450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512648 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.512727 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.512744 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.512829 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.512901 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":65,"thread_start_us":94,"threads_started":1}
I20260924 16:07:42.513119 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.507894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.13.8.65:44205)
0924 16:07:42.508184 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.508200 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.508241 (+ 41us) server_negotiation.cc:407] Connection header received
0924 16:07:42.508311 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.508316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.508500 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.508647 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:42.508877 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.509903 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.511584 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512353 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512441 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.512589 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.512876 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.512967 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260924 16:07:42.513458 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.507481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42575 (local address 127.0.0.1:35380)
0924 16:07:42.508547 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.508558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.508653 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.509246 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.509249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.509266 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:42.509363 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.509369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.510512 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.510515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.511662 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.511669 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512456 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.512565 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.512605 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.512735 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.512853 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":22,"thread_start_us":84,"threads_started":1}
I20260924 16:07:42.513657 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.506968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.13.8.67:44789)
0924 16:07:42.507313 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.507325 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.507818 (+ 493us) server_negotiation.cc:407] Connection header received
0924 16:07:42.507898 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.507904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.508116 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.508278 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:42.508597 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.509761 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.511424 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512316 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512441 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.512585 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.513419 (+ 834us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.513517 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":112,"threads_started":1}
I20260924 16:07:42.513360 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.506801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:44789 (local address 127.0.0.1:54916)
0924 16:07:42.507475 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.507527 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.507662 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.508285 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.508292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.508315 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:42.508449 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.508455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.509903 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.509908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.511298 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.511308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.512965 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.512968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.513061 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.513089 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.513166 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.513247 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":25,"thread_start_us":84,"threads_started":1}
I20260924 16:07:42.516295 23789 tablet_service.cc:1467] Processing CreateTablet for tablet 09ed31dc4865428bacc415b8bda312d9 (DEFAULT_TABLE table=test-table [id=710db9176b004a02b211ef7ad7cb5516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:42.517396 23789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ed31dc4865428bacc415b8bda312d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:42.516984 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 09ed31dc4865428bacc415b8bda312d9 (DEFAULT_TABLE table=test-table [id=710db9176b004a02b211ef7ad7cb5516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:42.516984 23925 tablet_service.cc:1467] Processing CreateTablet for tablet 09ed31dc4865428bacc415b8bda312d9 (DEFAULT_TABLE table=test-table [id=710db9176b004a02b211ef7ad7cb5516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:07:42.518047 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ed31dc4865428bacc415b8bda312d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:42.518038 23925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ed31dc4865428bacc415b8bda312d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:07:42.523656 24144 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: Bootstrap starting.
I20260924 16:07:42.525951 24144 tablet_bootstrap.cc:654] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:42.526544 24145 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Bootstrap starting.
I20260924 16:07:42.526793 24144 log.cc:824] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:42.527489 24146 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Bootstrap starting.
I20260924 16:07:42.528651 24145 tablet_bootstrap.cc:654] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:42.529537 24146 tablet_bootstrap.cc:654] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Neither blocks nor log segments found. Creating new log.
I20260924 16:07:42.529733 24145 log.cc:824] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:42.530304 24146 log.cc:824] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Log is configured to *not* fsync() on all Append() calls
I20260924 16:07:42.531756 24144 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: No bootstrap required, opened a new log
I20260924 16:07:42.531778 24145 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: No bootstrap required, opened a new log
I20260924 16:07:42.531963 24144 ts_tablet_manager.cc:1397] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260924 16:07:42.531970 24145 ts_tablet_manager.cc:1397] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:07:42.532150 24146 tablet_bootstrap.cc:492] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: No bootstrap required, opened a new log
I20260924 16:07:42.532315 24146 ts_tablet_manager.cc:1397] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260924 16:07:42.535873 24144 raft_consensus.cc:348] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.536108 24144 raft_consensus.cc:374] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:42.536209 24144 raft_consensus.cc:729] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa94b9adca4410ca3451eda6af035b5, State: Initialized, Role: FOLLOWER
I20260924 16:07:42.536039 24146 raft_consensus.cc:348] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.536371 24146 raft_consensus.cc:374] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:42.536235 24145 raft_consensus.cc:348] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.536432 24146 raft_consensus.cc:729] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4252f61b4cb940c393fc5e49a05309cf, State: Initialized, Role: FOLLOWER
I20260924 16:07:42.536445 24145 raft_consensus.cc:374] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:07:42.536491 24145 raft_consensus.cc:729] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d9f729505de4e07a840d0cbc64616ec, State: Initialized, Role: FOLLOWER
I20260924 16:07:42.536754 24144 consensus_queue.cc:260] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.536957 24146 consensus_queue.cc:260] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.536957 24145 consensus_queue.cc:260] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.537628 24127 heartbeater.cc:503] Master 127.13.8.126:36659 was elected leader, sending a full tablet report...
I20260924 16:07:42.537838 23992 heartbeater.cc:503] Master 127.13.8.126:36659 was elected leader, sending a full tablet report...
I20260924 16:07:42.537875 24144 ts_tablet_manager.cc:1428] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260924 16:07:42.537876 24145 ts_tablet_manager.cc:1428] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:07:42.538156 23855 heartbeater.cc:503] Master 127.13.8.126:36659 was elected leader, sending a full tablet report...
I20260924 16:07:42.538770 24146 ts_tablet_manager.cc:1428] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:07:42.550411 24152 raft_consensus.cc:482] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:07:42.550616 24152 raft_consensus.cc:504] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.551735 24152 leader_election.cc:288] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d9f729505de4e07a840d0cbc64616ec (127.13.8.67:44789), 8aa94b9adca4410ca3451eda6af035b5 (127.13.8.65:44205)
I20260924 16:07:42.556185 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.551961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:44789 (local address 127.13.8.66:33279)
0924 16:07:42.552071 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.552086 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.552175 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.552636 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.552642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.552687 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:42.552830 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.552836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.553763 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.553767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.555099 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.555109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.555750 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.555860 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.555887 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.555963 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.556045 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":3,"spinlock_wait_cycles":1792}
I20260924 16:07:42.556200 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.552285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:33279 (local address 127.13.8.67:44789)
0924 16:07:42.552402 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.552405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.552417 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:07:42.552459 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.552462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.552498 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.552597 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:42.552948 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.553664 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555230 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.555623 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555709 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.555787 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.555995 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.556080 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:07:42.556268 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.551969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44205 (local address 127.13.8.66:38515)
0924 16:07:42.552612 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.552625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:07:42.552709 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:07:42.552980 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:07:42.552983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:07:42.553000 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:07:42.553105 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.553111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.553942 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.553945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.555113 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:07:42.555120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.555821 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:07:42.555921 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.555968 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 16:07:42.556048 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 16:07:42.556114 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":284,"thread_start_us":160,"threads_started":1}
I20260924 16:07:42.556303 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.552111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:38515 (local address 127.13.8.65:44205)
0924 16:07:42.552257 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.552261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.552710 (+ 449us) server_negotiation.cc:407] Connection header received
0924 16:07:42.552826 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.552828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.552872 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.552938 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:07:42.553222 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.553845 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555259 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.555689 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.555796 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.555858 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.556084 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.556166 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:07:42.556831 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ed31dc4865428bacc415b8bda312d9" candidate_uuid: "4252f61b4cb940c393fc5e49a05309cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa94b9adca4410ca3451eda6af035b5" is_pre_election: true
I20260924 16:07:42.556975 24081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ed31dc4865428bacc415b8bda312d9" candidate_uuid: "4252f61b4cb940c393fc5e49a05309cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d9f729505de4e07a840d0cbc64616ec" is_pre_election: true
I20260924 16:07:42.557120 23809 raft_consensus.cc:2454] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4252f61b4cb940c393fc5e49a05309cf in term 0.
I20260924 16:07:42.557305 24081 raft_consensus.cc:2454] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4252f61b4cb940c393fc5e49a05309cf in term 0.
I20260924 16:07:42.557593 23877 leader_election.cc:302] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4252f61b4cb940c393fc5e49a05309cf, 8aa94b9adca4410ca3451eda6af035b5; no voters:
I20260924 16:07:42.557880 24152 raft_consensus.cc:2781] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:07:42.557976 24152 raft_consensus.cc:482] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:07:42.558033 24152 raft_consensus.cc:3037] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:42.559128 24152 raft_consensus.cc:504] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.559502 24152 leader_election.cc:288] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [CANDIDATE]: Term 1 election: Requested vote from peers 1d9f729505de4e07a840d0cbc64616ec (127.13.8.67:44789), 8aa94b9adca4410ca3451eda6af035b5 (127.13.8.65:44205)
I20260924 16:07:42.559857 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ed31dc4865428bacc415b8bda312d9" candidate_uuid: "4252f61b4cb940c393fc5e49a05309cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa94b9adca4410ca3451eda6af035b5"
I20260924 16:07:42.559863 24081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ed31dc4865428bacc415b8bda312d9" candidate_uuid: "4252f61b4cb940c393fc5e49a05309cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d9f729505de4e07a840d0cbc64616ec"
I20260924 16:07:42.559998 24081 raft_consensus.cc:3037] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:42.559988 23809 raft_consensus.cc:3037] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:07:42.561053 24081 raft_consensus.cc:2454] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4252f61b4cb940c393fc5e49a05309cf in term 1.
I20260924 16:07:42.561053 23809 raft_consensus.cc:2454] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4252f61b4cb940c393fc5e49a05309cf in term 1.
I20260924 16:07:42.561337 23880 leader_election.cc:302] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d9f729505de4e07a840d0cbc64616ec, 4252f61b4cb940c393fc5e49a05309cf; no voters:
I20260924 16:07:42.561514 24152 raft_consensus.cc:2781] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:07:42.561738 24152 raft_consensus.cc:686] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 1 LEADER]: Becoming Leader. State: Replica: 4252f61b4cb940c393fc5e49a05309cf, State: Running, Role: LEADER
I20260924 16:07:42.562071 24152 consensus_queue.cc:237] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } }
I20260924 16:07:42.565305 23662 catalog_manager.cc:5495] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4252f61b4cb940c393fc5e49a05309cf (127.13.8.66). New cstate: current_term: 1 leader_uuid: "4252f61b4cb940c393fc5e49a05309cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4252f61b4cb940c393fc5e49a05309cf" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 42575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 } health_report { overall_health: UNKNOWN } } }
I20260924 16:07:42.590090 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.586919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.13.8.66:42575)
0924 16:07:42.587048 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:07:42.587051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:07:42.587073 (+ 22us) server_negotiation.cc:407] Connection header received
0924 16:07:42.587149 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:07:42.587150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:07:42.587202 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:07:42.587289 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:07:42.587525 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.587997 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.588797 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:07:42.589004 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:07:42.589093 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:07:42.589344 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:07:42.589773 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:07:42.589863 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 16:07:42.589916 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 16:07:42.589977 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:07:42.596114 24081 raft_consensus.cc:1261] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 1 FOLLOWER]: Refusing update from remote peer 4252f61b4cb940c393fc5e49a05309cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:42.596114 23809 raft_consensus.cc:1261] T 09ed31dc4865428bacc415b8bda312d9 P 8aa94b9adca4410ca3451eda6af035b5 [term 1 FOLLOWER]: Refusing update from remote peer 4252f61b4cb940c393fc5e49a05309cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:07:42.596899 24154 consensus_queue.cc:1035] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aa94b9adca4410ca3451eda6af035b5" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:42.597114 24152 consensus_queue.cc:1035] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d9f729505de4e07a840d0cbc64616ec" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:07:42.605329 24157 mvcc.cc:205] Tried to move back new op lower bound from 7332929792386306048 to 7332929792256798720. Current Snapshot: MvccSnapshot[applied={T|T < 7332929792386306048}]
I20260924 16:07:42.605607 24159 mvcc.cc:205] Tried to move back new op lower bound from 7332929792386306048 to 7332929792256798720. Current Snapshot: MvccSnapshot[applied={T|T < 7332929792386306048}]
I20260924 16:07:42.608948 24158 mvcc.cc:205] Tried to move back new op lower bound from 7332929792386306048 to 7332929792256798720. Current Snapshot: MvccSnapshot[applied={T|T < 7332929792386306048}]
I20260924 16:07:42.614198 23662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:42.614428 23662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:07:42.616606 23662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35790270da4a43c4b8a903021102c6e9: Sending DeleteTablet for 3 replicas of tablet 09ed31dc4865428bacc415b8bda312d9
I20260924 16:07:42.617478 23789 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ed31dc4865428bacc415b8bda312d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:42 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:56042
I20260924 16:07:42.617478 23925 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ed31dc4865428bacc415b8bda312d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:42 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:35380
I20260924 16:07:42.617511 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ed31dc4865428bacc415b8bda312d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:07:42 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:54916
I20260924 16:07:42.618160 24169 tablet_replica.cc:330] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: stopping tablet replica
I20260924 16:07:42.618520 24169 raft_consensus.cc:2229] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:07:42.618649 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23726
I20260924 16:07:42.618841 24169 raft_consensus.cc:2258] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:42.619189 24168 tablet_replica.cc:330] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: stopping tablet replica
I20260924 16:07:42.619454 24168 raft_consensus.cc:2229] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:07:42.619786 24168 raft_consensus.cc:2258] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:07:42.622718 24169 ts_tablet_manager.cc:1905] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:42.623485 24168 ts_tablet_manager.cc:1905] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:07:42.626407 24168 ts_tablet_manager.cc:1918] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:42.626471 24169 ts_tablet_manager.cc:1918] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:07:42.626641 24169 log.cc:1196] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/09ed31dc4865428bacc415b8bda312d9
I20260924 16:07:42.626644 24168 log.cc:1196] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/09ed31dc4865428bacc415b8bda312d9
I20260924 16:07:42.627063 24168 ts_tablet_manager.cc:1939] T 09ed31dc4865428bacc415b8bda312d9 P 4252f61b4cb940c393fc5e49a05309cf: Deleting consensus metadata
I20260924 16:07:42.627154 24169 ts_tablet_manager.cc:1939] T 09ed31dc4865428bacc415b8bda312d9 P 1d9f729505de4e07a840d0cbc64616ec: Deleting consensus metadata
W20260924 16:07:42.627211 23647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:44205 (error 108)
I20260924 16:07:42.628123 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23860
I20260924 16:07:42.629251 23650 catalog_manager.cc:4841] TS 1d9f729505de4e07a840d0cbc64616ec (127.13.8.67:44789): tablet 09ed31dc4865428bacc415b8bda312d9 (table test-table [id=710db9176b004a02b211ef7ad7cb5516]) successfully deleted
I20260924 16:07:42.630473 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.630193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44205 (local address 127.0.0.1:56054)
0924 16:07:42.630336 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.630404 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260924 16:07:42.630678 23647 catalog_manager.cc:4584] TS 8aa94b9adca4410ca3451eda6af035b5 (127.13.8.65:44205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09ed31dc4865428bacc415b8bda312d9: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44205: connect: Connection refused (error 111)
I20260924 16:07:42.635172 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23997
I20260924 16:07:42.636969 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:07:42.635317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:42575 (local address 127.0.0.1:35398)
0924 16:07:42.636798 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0924 16:07:42.636864 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.66:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1374}
I20260924 16:07:42.641563 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 23631
2026-09-24T16:07:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2265 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[24173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[24173](info): set up 2 sockets
Sep 24 16:07:42 dist-test-slave-vj6n krb5kdc[24173](info): commencing operation
krb5kdc: starting...
W20260924 16:07:43.692279 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:07:43 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:07:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangeradmin_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangeradmin_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangerlookup_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangerlookup_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
I20260924 16:07:43.858951 13345 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskf9hPi5/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/postgres -l logfile start
2026-09-24 16:07:46.509 UTC [24204] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 16:07:46.509 UTC [24204] LOG: listening on IPv4 address "127.13.8.84", port 58919
2026-09-24 16:07:46.512 UTC [24204] LOG: listening on Unix socket "/tmp/.s.PGSQL.58919"
2026-09-24 16:07:46.547 UTC [24207] LOG: database system was shut down at 2026-09-24 16:07:45 UTC
2026-09-24 16:07:46.551 UTC [24204] LOG: database system is ready to accept connections
I20260924 16:07:47.483703 13345 mini_postgres.cc:96] Postgres bound to 58919
2026-09-24 16:07:47.491 UTC [24215] FATAL: database "slave" does not exist
127.13.8.84:58919 - accepting connections
I20260924 16:07:47.491854 13345 mini_ranger.cc:161] Starting Ranger...
I20260924 16:07:47.509457 13345 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 16:07:48.008751 13345 mini_ranger.cc:87] Created ranger Postgres database
I20260924 16:07:48.008853 13345 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin
2026-09-24 16:07:48,488 [I] DB FLAVOR :POSTGRES
2026-09-24 16:07:48,489 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 16:07:48,489 [I] Checking connection..
2026-09-24 16:07:48,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 16:07:48,764 [I] Checking connection passed.
2026-09-24 16:07:48,765 [I] --------- Verifying version history table ---------
2026-09-24 16:07:48,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:07:49,013 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 16:07:49,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:07:49,258 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 16:07:49,259 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 16:07:49,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 16:07:49.489 UTC [24279] WARNING: there is no transaction in progress
2026-09-24 16:07:49,496 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 16:07:49,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:07:49,757 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 16:07:49,758 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 16:07:49,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 16:07:50,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 16:07:50,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vj6n.c.gcp-upstream.internal','N') ;"
2026-09-24 16:07:50,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 16:07:50,759 [I] Table x_portal_user does not exist in database ranger
2026-09-24 16:07:50,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 16:07:51,019 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 16:07:51,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 16:07:51,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 16:07:51,535 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 16:07:51,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 16:07:52.517 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.533 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.549 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.565 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.581 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.600 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.653 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.660 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.675 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.685 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.695 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.709 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.719 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.725 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.733 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:52.740 UTC [24388] WARNING: there is no transaction in progress
2026-09-24 16:07:53,394 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 16:07:53,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 16:07:53,616 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 16:07:53,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 16:07:53,861 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 16:07:53,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 16:07:54,096 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 16:07:54,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 16:07:54,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 16:07:54,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vj6n.c.gcp-upstream.internal';"
2026-09-24 16:07:54,843 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 16:07:54,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 16:07:55,101 [I] DB_PATCHES have already been applied
I20260924 16:07:55.106959 13345 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskf9hPi5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 16:07:55.107017 13345 mini_ranger.cc:193] Using host: 127.13.8.84
I20260924 16:07:55.110596 13345 mini_ranger.cc:227] Ranger admin URL: http://127.13.8.84:44137
Sep 24, 2026 4:07:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 4:07:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 4:07:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 4:07:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 4:07:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 4:07:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 4:07:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 16:07:57.405488 13345 mini_ranger.cc:160] Time spent starting Ranger: real 9.914s user 0.000s sys 0.012s
Sep 24, 2026 4:08:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 4:08:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 4:08:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 4:08:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 4:08:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 16:08:20.174691 13345 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangerkms_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/rangerkms_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 16:08:20 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266100, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 16:08:20.268666 13345 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 16:08:20.278057 13345 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 16:08:21.101658 13345 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 16:08:21.101763 13345 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms
I20260924 16:08:21.101871 13345 mini_ranger_kms.cc:227] Using postgres at 127.13.8.84:58919
2026-09-24 16:08:21,938 [I] DB FLAVOR :POSTGRES
2026-09-24 16:08:21,939 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 16:08:21,939 [I] Checking connection
2026-09-24 16:08:21,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 16:08:22,262 [I] connection success
2026-09-24 16:08:22,262 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 16:08:22,262 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 16:08:22,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 16:08:22,524 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 16:08:22,525 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 16:08:22,525 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 16:08:22,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:58919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 16:08:22,778 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 16:08:23.213352 13345 mini_ranger_kms.cc:310] Created kms service
I20260924 16:08:23.384610 13345 mini_ranger_kms.cc:326] Created kudu user
I20260924 16:08:23.427389 13345 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 16:08:23 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266100, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:23.842607 13345 mini_ranger_kms.cc:384] Added ranger policy
I20260924 16:08:23.842782 13345 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms:/tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 16:08:23.842830 13345 mini_ranger_kms.cc:240] Using host: 127.13.8.84
I20260924 16:08:23.846673 13345 mini_ranger_kms.cc:276] Ranger KMS PID: 24571
I20260924 16:08:23.846809 13345 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.8.84:44059
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 4:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 16:08:26.187408 13345 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.919s user 0.017s sys 0.481s
I20260924 16:08:26.187515 13345 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 16:08:26.187664 13345 mini_ranger_kms.cc:401] 127.13.8.84:44059/kms/v1/keys
Sep 24 16:08:26 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790266106, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 16:08:26 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266106, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
Sep 24 16:08:27 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266106, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
Sep 24 16:08:29 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:08:29.689524 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43053
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:43053
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:44059/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:08:29.823450 24622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:08:29.823726 24622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:08:29.823766 24622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:08:29.827986 24622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:08:29.828038 24622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:08:29.828058 24622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:08:29.828095 24622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:08:29.828125 24622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:08:29.833606 24622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.8.84:44059/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:43053
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:43053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:08:29.834712 24622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:08:29.835739 24622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:08:29.845077 24627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:29.845105 24628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:29.845371 24622 server_base.cc:1034] running on GCE node
W20260924 16:08:29.845422 24631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:29.846027 24622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:08:29.847154 24622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:08:29.848342 24622 hybrid_clock.cc:648] HybridClock initialized: now 1790266109848314 us; error 44 us; skew 500 ppm
Sep 24 16:08:29 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266109, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:08:29.856804 24622 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:08:29.857942 24622 webserver.cc:460] Webserver started at http://127.13.8.126:37541/ using document root <none> and password file <none>
I20260924 16:08:29.858666 24622 fs_manager.cc:359] Metadata directory not provided
I20260924 16:08:29.858742 24622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:08:29.858971 24622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:08:29 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266109, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:29.916909 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "e884d19634a44dbeae1e0fbd535c9192"
format_stamp: "Formatted at 2026-09-24 16:08:29 on dist-test-slave-vj6n"
server_key: "64e570abe5b1af5d7277b85a34d6ac06"
server_key_iv: "a04789aece4ab1e023f537f44623c846"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:29.917667 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "e884d19634a44dbeae1e0fbd535c9192"
format_stamp: "Formatted at 2026-09-24 16:08:29 on dist-test-slave-vj6n"
server_key: "64e570abe5b1af5d7277b85a34d6ac06"
server_key_iv: "a04789aece4ab1e023f537f44623c846"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:29.922118 24622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260924 16:08:29.941166 24638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:29.943008 24622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:08:29.943156 24622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "e884d19634a44dbeae1e0fbd535c9192"
format_stamp: "Formatted at 2026-09-24 16:08:29 on dist-test-slave-vj6n"
server_key: "64e570abe5b1af5d7277b85a34d6ac06"
server_key_iv: "a04789aece4ab1e023f537f44623c846"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:29.943295 24622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:08:29.981606 24622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:08:29.985424 24622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:08:29.985635 24622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:08:29.996912 24622 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:43053
I20260924 16:08:29.996951 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:43053 every 8 connection(s)
I20260924 16:08:29.998272 24622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:08:29.998816 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 24622
I20260924 16:08:29.998992 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:08:29.999395 13345 external_mini_cluster.cc:1428] Setting key 4ecf5a81cf9b8577585d92701efc862c
I20260924 16:08:30.002104 24691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:08:30.009832 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: Bootstrap starting.
Sep 24 16:08:30 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:08:30.013211 24691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: Neither blocks nor log segments found. Creating new log.
I20260924 16:08:30.014825 24691 log.cc:824] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: Log is configured to *not* fsync() on all Append() calls
I20260924 16:08:30.016911 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:30.001292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.13.8.126:43053)
0924 16:08:30.001822 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:30.001841 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:30.001887 (+ 46us) server_negotiation.cc:407] Connection header received
0924 16:08:30.002792 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:30.002816 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:30.003245 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:30.003606 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:08:30.004435 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:30.006054 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:30.007274 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:30.008045 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:30.008206 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:30.011425 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:08:30.011433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:08:30.011451 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:08:30.011489 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:08:30.013725 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:30.014254 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:30.014261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:30.014268 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:30.014403 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:30.014714 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:30.014718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:30.014719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:30.015276 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:08:30.015504 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:30.015821 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 16:08:30.016159 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":202,"threads_started":1}
I20260924 16:08:30.017361 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: No bootstrap required, opened a new log
I20260924 16:08:30.021893 24691 raft_consensus.cc:348] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } }
I20260924 16:08:30.022197 24691 raft_consensus.cc:374] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:08:30.022306 24691 raft_consensus.cc:729] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e884d19634a44dbeae1e0fbd535c9192, State: Initialized, Role: FOLLOWER
I20260924 16:08:30.022797 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } }
I20260924 16:08:30.022957 24691 raft_consensus.cc:388] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:08:30.023032 24691 raft_consensus.cc:482] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:08:30.023130 24691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:08:30.024350 24691 raft_consensus.cc:504] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } }
I20260924 16:08:30.024721 24691 leader_election.cc:302] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e884d19634a44dbeae1e0fbd535c9192; no voters:
I20260924 16:08:30.025098 24691 leader_election.cc:288] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:08:30.025352 24696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:08:30.025827 24696 raft_consensus.cc:686] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [term 1 LEADER]: Becoming Leader. State: Replica: e884d19634a44dbeae1e0fbd535c9192, State: Running, Role: LEADER
I20260924 16:08:30.026198 24696 consensus_queue.cc:237] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } }
I20260924 16:08:30.026629 24691 sys_catalog.cc:564] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:08:30.028151 24698 sys_catalog.cc:455] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e884d19634a44dbeae1e0fbd535c9192. Latest consensus state: current_term: 1 leader_uuid: "e884d19634a44dbeae1e0fbd535c9192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } } }
I20260924 16:08:30.028278 24698 sys_catalog.cc:458] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [sys.catalog]: This master's current role is: LEADER
I20260924 16:08:30.028672 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e884d19634a44dbeae1e0fbd535c9192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e884d19634a44dbeae1e0fbd535c9192" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 43053 } } }
I20260924 16:08:30.028753 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192 [sys.catalog]: This master's current role is: LEADER
I20260924 16:08:30.029080 24691 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 16:08:30.029790 24709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:08:30.037719 24709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:08:30.048056 24709 catalog_manager.cc:1347] Generated new cluster ID: 7e4fa4670f3a4dad80987fc5554045fd
I20260924 16:08:30.048182 24709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:08:30.069180 24709 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:08:30.070992 24709 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:08:30.086349 24709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: Generated new TSK 0
I20260924 16:08:30.087172 24709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:08:30.857533 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:44059/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:08:31.023478 24737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:08:31.023721 24737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:08:31.023782 24737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:08:31.028194 24737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:08:31.028251 24737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:08:31.028334 24737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:08:31.033939 24737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.8.84:44059/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:08:31.035182 24737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:08:31.036564 24737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:08:31.050014 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:31.050083 24747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:31.050083 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:31.051682 24737 server_base.cc:1034] running on GCE node
I20260924 16:08:31.052109 24737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:08:31.052755 24737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:08:31.053934 24737 hybrid_clock.cc:648] HybridClock initialized: now 1790266111053903 us; error 52 us; skew 500 ppm
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:08:31.057364 24737 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:08:31.058692 24737 webserver.cc:460] Webserver started at http://127.13.8.65:41749/ using document root <none> and password file <none>
I20260924 16:08:31.059526 24737 fs_manager.cc:359] Metadata directory not provided
I20260924 16:08:31.059646 24737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:08:31.059998 24737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:31.084275 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "a1d76cbb30b445f89853d70c24d4268e"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "d9f923e2e2f1840f6fd480ec72b3ed47"
server_key_iv: "0a1f5f1ecefbba05a21671d022e4fdcc"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.085312 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "a1d76cbb30b445f89853d70c24d4268e"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "d9f923e2e2f1840f6fd480ec72b3ed47"
server_key_iv: "0a1f5f1ecefbba05a21671d022e4fdcc"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.091439 24737 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260924 16:08:31.109956 24754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.111838 24737 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260924 16:08:31.112139 24737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "a1d76cbb30b445f89853d70c24d4268e"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "d9f923e2e2f1840f6fd480ec72b3ed47"
server_key_iv: "0a1f5f1ecefbba05a21671d022e4fdcc"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.112392 24737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:08:31.129441 24737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:08:31.133391 24737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:08:31.133677 24737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:08:31.134644 24737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:08:31.135901 24737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:08:31.136034 24737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.136196 24737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:08:31.136289 24737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.154021 24737 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43957
I20260924 16:08:31.155910 24737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:08:31.157472 24868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43957 every 8 connection(s)
I20260924 16:08:31.161298 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 24737
I20260924 16:08:31.161425 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:08:31.161767 13345 external_mini_cluster.cc:1428] Setting key f3d309c8c8dbae2545feaac65899c76d
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:08:31.178929 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.162284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59045 (local address 127.13.8.126:43053)
0924 16:08:31.165052 (+ 2768us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:31.165057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:31.165072 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:08:31.165115 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:31.165118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:31.165166 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:31.165259 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:08:31.167022 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.167971 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.169830 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.170294 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.170379 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.173751 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:08:31.173756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:08:31.173759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:08:31.173788 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:08:31.176019 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.176747 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.176751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.176753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.176874 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.177347 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.177351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.177353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.177609 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:08:31.177694 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:31.178549 (+ 855us) server_negotiation.cc:299] Negotiation successful
0924 16:08:31.178694 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2681,"thread_start_us":89,"threads_started":1}
I20260924 16:08:31.181025 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.160969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43053 (local address 127.13.8.65:59045)
0924 16:08:31.163288 (+ 2319us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:31.163367 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:31.164566 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:31.165605 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:31.165625 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:31.166019 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:08:31.166820 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.166840 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.168149 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.168153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.169676 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.169688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.170627 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.170631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.170842 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.171614 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:08:31.171638 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:08:31.173539 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:08:31.176197 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.176204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.176223 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.176612 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.177024 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.177028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.177030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.177225 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.177792 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:08:31.177800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:08:31.178193 (+ 393us) client_negotiation.cc:769] Sending connection context
0924 16:08:31.179467 (+ 1274us) client_negotiation.cc:240] Negotiation successful
0924 16:08:31.179847 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2021,"thread_start_us":155,"threads_started":1}
I20260924 16:08:31.183368 24869 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43053
I20260924 16:08:31.183723 24869 heartbeater.cc:464] Registering TS with master...
I20260924 16:08:31.184386 24869 heartbeater.cc:511] Master 127.13.8.126:43053 requested a full tablet report, sending...
I20260924 16:08:31.186497 24655 ts_manager.cc:194] Registered new tserver with Master: a1d76cbb30b445f89853d70c24d4268e (127.13.8.65:43957)
I20260924 16:08:31.188431 24655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:59045
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:08:31.225757 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:44059/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 16:08:31.388456 24877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:08:31.388828 24877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:08:31.388895 24877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:08:31.395047 24877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:08:31.395169 24877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:08:31.395282 24877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:08:31.403189 24877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.8.84:44059/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:08:31.404558 24877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:08:31.405800 24877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:08:31.418948 24884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:31.419131 24877 server_base.cc:1034] running on GCE node
W20260924 16:08:31.419322 24886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:31.419586 24883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:31.420105 24877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:08:31.420846 24877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:08:31.422044 24877 hybrid_clock.cc:648] HybridClock initialized: now 1790266111422015 us; error 44 us; skew 500 ppm
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:08:31.425946 24877 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:08:31.427161 24877 webserver.cc:460] Webserver started at http://127.13.8.66:39049/ using document root <none> and password file <none>
I20260924 16:08:31.427956 24877 fs_manager.cc:359] Metadata directory not provided
I20260924 16:08:31.428062 24877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:08:31.428390 24877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:31.449143 24877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "a204237fdb4e4aedadf455fdcc7f3b66"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "f2540b025a794a2c7d19b348d3a2ea63"
server_key_iv: "ae09ba3f5514cf2bc01f4a314a0cbc8b"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.450114 24877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "a204237fdb4e4aedadf455fdcc7f3b66"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "f2540b025a794a2c7d19b348d3a2ea63"
server_key_iv: "ae09ba3f5514cf2bc01f4a314a0cbc8b"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.455979 24877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 16:08:31.490559 24893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.492542 24877 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260924 16:08:31.492699 24877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "a204237fdb4e4aedadf455fdcc7f3b66"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "f2540b025a794a2c7d19b348d3a2ea63"
server_key_iv: "ae09ba3f5514cf2bc01f4a314a0cbc8b"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.492853 24877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:08:31.509974 24877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:08:31.513870 24877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:08:31.514129 24877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:08:31.515098 24877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:08:31.516566 24877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:08:31.516654 24877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.516723 24877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:08:31.516757 24877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.535967 24877 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:35287
I20260924 16:08:31.537460 24877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:08:31.543646 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 24877
I20260924 16:08:31.543780 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:08:31.544103 13345 external_mini_cluster.cc:1428] Setting key d87e21287053600657339962f988c049
I20260924 16:08:31.549275 25007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:35287 every 8 connection(s)
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:08:31.558598 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.541910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:46055 (local address 127.13.8.126:43053)
0924 16:08:31.542060 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:31.542064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:31.542083 (+ 19us) server_negotiation.cc:407] Connection header received
0924 16:08:31.543201 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:31.543204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:31.543258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:31.543312 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:08:31.545143 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.545972 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.547897 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.548265 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.548345 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.553118 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:08:31.553125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:08:31.553129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:08:31.553166 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:08:31.555520 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.556162 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.556166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.556168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.556271 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.556784 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.556788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.556790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.557165 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:08:31.557319 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:31.558258 (+ 939us) server_negotiation.cc:299] Negotiation successful
0924 16:08:31.558412 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:08:31.560950 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.540619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43053 (local address 127.13.8.66:46055)
0924 16:08:31.541380 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:31.541444 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:31.542915 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:31.543777 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:31.543787 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:31.544277 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:08:31.544952 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.544971 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.546145 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.546148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.547753 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.547765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.548550 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.548554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.548754 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.549509 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:08:31.549529 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:08:31.552824 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:08:31.555670 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.555676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.555695 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.556036 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.556379 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.556381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.556383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.556593 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.557393 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:08:31.557400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:08:31.557876 (+ 476us) client_negotiation.cc:769] Sending connection context
0924 16:08:31.559254 (+ 1378us) client_negotiation.cc:240] Negotiation successful
0924 16:08:31.559742 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:08:31.562696 25008 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43053
I20260924 16:08:31.562999 25008 heartbeater.cc:464] Registering TS with master...
I20260924 16:08:31.563777 25008 heartbeater.cc:511] Master 127.13.8.126:43053 requested a full tablet report, sending...
I20260924 16:08:31.565092 24655 ts_manager.cc:194] Registered new tserver with Master: a204237fdb4e4aedadf455fdcc7f3b66 (127.13.8.66:35287)
I20260924 16:08:31.565882 24655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:46055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:08:31.586184 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:44059/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:08:31.771382 25013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:08:31.771802 25013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:08:31.771875 25013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:08:31.779151 25013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:08:31.779233 25013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:08:31.779330 25013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:31.789067 25013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.8.84:44059/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:08:31.790640 25013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:08:31.792101 25013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:08:31.806876 25019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:31.806856 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:08:31.807570 25018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:08:31.808964 25013 server_base.cc:1034] running on GCE node
I20260924 16:08:31.809525 25013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:08:31.810459 25013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:08:31.811666 25013 hybrid_clock.cc:648] HybridClock initialized: now 1790266111811589 us; error 90 us; skew 500 ppm
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:08:31.815881 25013 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:08:31.817098 25013 webserver.cc:460] Webserver started at http://127.13.8.67:43503/ using document root <none> and password file <none>
I20260924 16:08:31.817843 25013 fs_manager.cc:359] Metadata directory not provided
I20260924 16:08:31.817934 25013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:08:31.818185 25013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:31.846818 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "157e611a4932460485c0ac85fca81a33"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "18fb9f93a129dbabb47671c16e599aef"
server_key_iv: "7aca8c4529607ed9dd8ddc56388e886c"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.847988 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "157e611a4932460485c0ac85fca81a33"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "18fb9f93a129dbabb47671c16e599aef"
server_key_iv: "7aca8c4529607ed9dd8ddc56388e886c"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.853497 25013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 16:08:31.877738 25028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.879622 25013 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 16:08:31.879752 25013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "157e611a4932460485c0ac85fca81a33"
format_stamp: "Formatted at 2026-09-24 16:08:31 on dist-test-slave-vj6n"
server_key: "18fb9f93a129dbabb47671c16e599aef"
server_key_iv: "7aca8c4529607ed9dd8ddc56388e886c"
server_key_version: "kuduclusterkey@0"
I20260924 16:08:31.879886 25013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:08:31.924510 25013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:08:31.928520 25013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:08:31.928738 25013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:08:31.929531 25013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:08:31.930655 25013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:08:31.930717 25013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:08:31.930773 25013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:08:31.930850 25013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260924 16:08:31.945616 25013 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:36927
I20260924 16:08:31.946411 25143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:36927 every 8 connection(s)
I20260924 16:08:31.946882 25013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:08:31.947816 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 25013
I20260924 16:08:31.947938 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:08:31.948334 13345 external_mini_cluster.cc:1428] Setting key 32d1b5b98b03f1819e5c5beb4473b0c5
Sep 24 16:08:31 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:08:31.974308 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.951973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:53355 (local address 127.13.8.126:43053)
0924 16:08:31.952144 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:31.952149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:31.955831 (+ 3682us) server_negotiation.cc:407] Connection header received
0924 16:08:31.957131 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:31.957134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:31.957188 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:31.957273 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:08:31.958735 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.959547 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.961273 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.961641 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.961732 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.969485 (+ 7753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:08:31.969489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:08:31.969492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:08:31.969522 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:08:31.971332 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.972059 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.972062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.972064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.972153 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.972685 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.972692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.972696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.972976 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:08:31.973105 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:31.973971 (+ 866us) server_negotiation.cc:299] Negotiation successful
0924 16:08:31.974102 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:08:31.975571 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.950901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:43053 (local address 127.13.8.67:53355)
0924 16:08:31.955587 (+ 4686us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:31.955658 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:31.956902 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:31.957498 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:31.957508 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:31.957873 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:08:31.958580 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.958600 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.959694 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.959697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.961096 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:31.961108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.961948 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.961954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:31.962165 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.967219 (+ 5054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:08:31.967245 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:08:31.969274 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:08:31.971485 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.971509 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.971529 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.971931 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.972269 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:08:31.972275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:08:31.972277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:08:31.972533 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:08:31.973194 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:08:31.973202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:08:31.973631 (+ 429us) client_negotiation.cc:769] Sending connection context
0924 16:08:31.973894 (+ 263us) client_negotiation.cc:240] Negotiation successful
0924 16:08:31.974330 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4395,"thread_start_us":121,"threads_started":1}
I20260924 16:08:31.976899 25144 heartbeater.cc:347] Connected to a master server at 127.13.8.126:43053
I20260924 16:08:31.977180 25144 heartbeater.cc:464] Registering TS with master...
I20260924 16:08:31.977783 25144 heartbeater.cc:511] Master 127.13.8.126:43053 requested a full tablet report, sending...
I20260924 16:08:31.979064 24655 ts_manager.cc:194] Registered new tserver with Master: 157e611a4932460485c0ac85fca81a33 (127.13.8.67:36927)
I20260924 16:08:31.979921 24655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:53355
I20260924 16:08:31.985584 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:08:31.999555 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:31.988180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.13.8.126:43053)
0924 16:08:31.988308 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:31.988312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:31.988702 (+ 390us) server_negotiation.cc:407] Connection header received
0924 16:08:31.988939 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:31.988954 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:31.989008 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:31.989068 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:08:31.990295 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.991062 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.992368 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:31.992751 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:31.992849 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:31.995161 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:08:31.995167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:08:31.995170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:08:31.995204 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:08:31.997162 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.997769 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.997775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.997777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.997889 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:08:31.998331 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:08:31.998335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:08:31.998337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:08:31.998623 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:08:31.998742 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:31.999190 (+ 448us) server_negotiation.cc:299] Negotiation successful
0924 16:08:31.999346 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:08:32.005838 24655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:08:32.010998 24655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:08:32.036886 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.028368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43957 (local address 127.0.0.1:38772)
0924 16:08:32.028882 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.028941 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:32.030867 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:32.031775 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:32.031781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:32.031807 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:08:32.031996 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.032003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.033681 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.033684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.035082 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.035095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.036274 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.036277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.036443 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.036500 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 16:08:32.036585 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:08:32.036699 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260924 16:08:32.038200 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.029328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.13.8.65:43957)
0924 16:08:32.031245 (+ 1917us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.031252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.031278 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:08:32.031359 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.031365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.031603 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.032167 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:08:32.032305 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.033546 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.035233 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.036108 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.037695 (+ 1587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.037849 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.037902 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.038014 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1780,"thread_start_us":65,"threads_started":1}
I20260924 16:08:32.039465 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.030968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:35287 (local address 127.0.0.1:40994)
0924 16:08:32.034328 (+ 3360us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.034344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:32.034430 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:32.035330 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:32.035333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:32.035350 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:08:32.035431 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.035437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.036808 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.036811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.037899 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.037909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.039081 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.039085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.039173 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.039198 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 16:08:32.039245 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 16:08:32.039311 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3287,"thread_start_us":156,"threads_started":1}
I20260924 16:08:32.040333 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.031493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.13.8.66:35287)
0924 16:08:32.034666 (+ 3173us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.034681 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.034705 (+ 24us) server_negotiation.cc:407] Connection header received
0924 16:08:32.034782 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.034787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.035013 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.035182 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:08:32.035558 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.036687 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.038071 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.038916 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.039891 (+ 975us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.040066 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.040116 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.040205 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3064,"thread_start_us":150,"threads_started":1}
I20260924 16:08:32.042027 24784 tablet_service.cc:1467] Processing CreateTablet for tablet 4363ee27a991440097248550539ae9e5 (DEFAULT_TABLE table=test-table [id=f3fe858bd82d412e9f82d8f424299590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:08:32.043506 24784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4363ee27a991440097248550539ae9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:08:32.043414 24935 tablet_service.cc:1467] Processing CreateTablet for tablet 4363ee27a991440097248550539ae9e5 (DEFAULT_TABLE table=test-table [id=f3fe858bd82d412e9f82d8f424299590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:08:32.044516 24935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4363ee27a991440097248550539ae9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:08:32.054198 25164 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Bootstrap starting.
I20260924 16:08:32.057047 25164 tablet_bootstrap.cc:654] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Neither blocks nor log segments found. Creating new log.
I20260924 16:08:32.057564 25163 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Bootstrap starting.
I20260924 16:08:32.060721 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.032331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:36927 (local address 127.0.0.1:44210)
0924 16:08:32.037495 (+ 5164us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.037508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:32.037573 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:32.054568 (+ 16995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:32.054571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:32.054596 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:08:32.054710 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.054717 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.057077 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.057082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.058231 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.058275 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.060231 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.060235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.060374 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.060407 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 16:08:32.060468 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 16:08:32.060569 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5097,"thread_start_us":85,"threads_started":1}
I20260924 16:08:32.061401 25164 log.cc:824] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:08:32.062196 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.032738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.13.8.67:36927)
0924 16:08:32.053983 (+ 21245us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.053998 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.054030 (+ 32us) server_negotiation.cc:407] Connection header received
0924 16:08:32.054122 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.054127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.054386 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.054884 (+ 498us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:08:32.055050 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.056241 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.058449 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.060054 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.061614 (+ 1560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.061798 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.061869 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.062020 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21098,"thread_start_us":243,"threads_started":1}
I20260924 16:08:32.063619 25163 tablet_bootstrap.cc:654] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Neither blocks nor log segments found. Creating new log.
I20260924 16:08:32.065021 25163 log.cc:824] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Log is configured to *not* fsync() on all Append() calls
I20260924 16:08:32.066496 25078 tablet_service.cc:1467] Processing CreateTablet for tablet 4363ee27a991440097248550539ae9e5 (DEFAULT_TABLE table=test-table [id=f3fe858bd82d412e9f82d8f424299590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:08:32.067786 25164 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: No bootstrap required, opened a new log
I20260924 16:08:32.067806 25078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4363ee27a991440097248550539ae9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:08:32.068101 25164 ts_tablet_manager.cc:1397] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.005s
I20260924 16:08:32.073581 25164 raft_consensus.cc:348] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.074092 25164 raft_consensus.cc:374] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:08:32.074235 25164 raft_consensus.cc:729] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d76cbb30b445f89853d70c24d4268e, State: Initialized, Role: FOLLOWER
I20260924 16:08:32.074909 25164 consensus_queue.cc:260] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.076488 25164 ts_tablet_manager.cc:1428] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260924 16:08:32.078496 25168 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Bootstrap starting.
I20260924 16:08:32.078960 25163 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: No bootstrap required, opened a new log
I20260924 16:08:32.079741 25163 ts_tablet_manager.cc:1397] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260924 16:08:32.082324 25168 tablet_bootstrap.cc:654] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Neither blocks nor log segments found. Creating new log.
I20260924 16:08:32.083618 25168 log.cc:824] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Log is configured to *not* fsync() on all Append() calls
I20260924 16:08:32.085688 25163 raft_consensus.cc:348] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.086581 24869 heartbeater.cc:503] Master 127.13.8.126:43053 was elected leader, sending a full tablet report...
I20260924 16:08:32.086793 25168 tablet_bootstrap.cc:492] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: No bootstrap required, opened a new log
I20260924 16:08:32.087013 25168 ts_tablet_manager.cc:1397] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260924 16:08:32.088025 25163 raft_consensus.cc:374] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:08:32.088171 25163 raft_consensus.cc:729] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a204237fdb4e4aedadf455fdcc7f3b66, State: Initialized, Role: FOLLOWER
I20260924 16:08:32.088800 25163 consensus_queue.cc:260] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.090637 25169 raft_consensus.cc:482] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:08:32.090910 25169 raft_consensus.cc:504] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.093376 25163 ts_tablet_manager.cc:1428] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260924 16:08:32.093791 25008 heartbeater.cc:503] Master 127.13.8.126:43053 was elected leader, sending a full tablet report...
I20260924 16:08:32.097263 25168 raft_consensus.cc:348] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.097759 25168 raft_consensus.cc:374] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:08:32.097883 25169 leader_election.cc:288] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a204237fdb4e4aedadf455fdcc7f3b66 (127.13.8.66:35287), 157e611a4932460485c0ac85fca81a33 (127.13.8.67:36927)
I20260924 16:08:32.098346 25168 raft_consensus.cc:729] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157e611a4932460485c0ac85fca81a33, State: Initialized, Role: FOLLOWER
I20260924 16:08:32.099114 25168 consensus_queue.cc:260] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.100622 25168 ts_tablet_manager.cc:1428] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260924 16:08:32.101082 25144 heartbeater.cc:503] Master 127.13.8.126:43053 was elected leader, sending a full tablet report...
I20260924 16:08:32.106163 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.093157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:40293 (local address 127.13.8.66:35287)
0924 16:08:32.093295 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.093301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.098629 (+ 5328us) server_negotiation.cc:407] Connection header received
0924 16:08:32.098992 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.098997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.099053 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.099108 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:08:32.102116 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.102875 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.104524 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.105245 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.105335 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.105410 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.105838 (+ 428us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.105951 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:08:32.107076 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.092631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:35287 (local address 127.13.8.65:40293)
0924 16:08:32.098439 (+ 5808us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.098458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:32.098827 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:32.099285 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:32.099288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:32.099421 (+ 133us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:08:32.100097 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.100108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.103078 (+ 2970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.103083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.104378 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.104389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.105520 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.105524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.105633 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.105658 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 16:08:32.106780 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0924 16:08:32.106864 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5723,"thread_start_us":189,"threads_started":1}
I20260924 16:08:32.108031 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4363ee27a991440097248550539ae9e5" candidate_uuid: "a1d76cbb30b445f89853d70c24d4268e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" is_pre_election: true
I20260924 16:08:32.108414 24959 raft_consensus.cc:2454] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d76cbb30b445f89853d70c24d4268e in term 0.
I20260924 16:08:32.109500 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.098011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:46181 (local address 127.13.8.67:36927)
0924 16:08:32.098181 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.098187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.103999 (+ 5812us) server_negotiation.cc:407] Connection header received
0924 16:08:32.104204 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.104208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.104259 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.104313 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:08:32.104924 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.105686 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.108119 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.108685 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.108789 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.108857 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.109264 (+ 407us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.109357 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:08:32.109633 24757 leader_election.cc:302] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1d76cbb30b445f89853d70c24d4268e, a204237fdb4e4aedadf455fdcc7f3b66; no voters:
I20260924 16:08:32.109951 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.097453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:36927 (local address 127.13.8.65:46181)
0924 16:08:32.103860 (+ 6407us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.103878 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:08:32.104079 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:08:32.104483 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:08:32.104489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:08:32.104512 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:08:32.104778 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.104787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.105962 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.105966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.107278 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:08:32.107293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.108971 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.108977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:08:32.109102 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.109128 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:08:32.109721 (+ 593us) client_negotiation.cc:240] Negotiation successful
0924 16:08:32.109805 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6328,"thread_start_us":164,"threads_started":1}
I20260924 16:08:32.110700 25167 raft_consensus.cc:2781] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:08:32.110762 25095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4363ee27a991440097248550539ae9e5" candidate_uuid: "a1d76cbb30b445f89853d70c24d4268e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157e611a4932460485c0ac85fca81a33" is_pre_election: true
I20260924 16:08:32.110853 25167 raft_consensus.cc:482] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:08:32.110918 25167 raft_consensus.cc:3037] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:08:32.111084 25095 raft_consensus.cc:2454] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d76cbb30b445f89853d70c24d4268e in term 0.
I20260924 16:08:32.112277 25167 raft_consensus.cc:504] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.113042 25095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4363ee27a991440097248550539ae9e5" candidate_uuid: "a1d76cbb30b445f89853d70c24d4268e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157e611a4932460485c0ac85fca81a33"
I20260924 16:08:32.113039 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4363ee27a991440097248550539ae9e5" candidate_uuid: "a1d76cbb30b445f89853d70c24d4268e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a204237fdb4e4aedadf455fdcc7f3b66"
I20260924 16:08:32.113168 25095 raft_consensus.cc:3037] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:08:32.113168 24959 raft_consensus.cc:3037] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:08:32.114812 24959 raft_consensus.cc:2454] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d76cbb30b445f89853d70c24d4268e in term 1.
I20260924 16:08:32.115187 25095 raft_consensus.cc:2454] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d76cbb30b445f89853d70c24d4268e in term 1.
I20260924 16:08:32.115240 24757 leader_election.cc:302] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1d76cbb30b445f89853d70c24d4268e, a204237fdb4e4aedadf455fdcc7f3b66; no voters:
I20260924 16:08:32.115689 25169 raft_consensus.cc:2781] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:08:32.116039 25169 raft_consensus.cc:686] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 1 LEADER]: Becoming Leader. State: Replica: a1d76cbb30b445f89853d70c24d4268e, State: Running, Role: LEADER
I20260924 16:08:32.116510 25169 consensus_queue.cc:237] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } }
I20260924 16:08:32.118589 25167 leader_election.cc:288] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [CANDIDATE]: Term 1 election: Requested vote from peers a204237fdb4e4aedadf455fdcc7f3b66 (127.13.8.66:35287), 157e611a4932460485c0ac85fca81a33 (127.13.8.67:36927)
I20260924 16:08:32.120762 24654 catalog_manager.cc:5495] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d76cbb30b445f89853d70c24d4268e (127.13.8.65). New cstate: current_term: 1 leader_uuid: "a1d76cbb30b445f89853d70c24d4268e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d76cbb30b445f89853d70c24d4268e" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 } health_report { overall_health: UNKNOWN } } }
Sep 24 16:08:32 dist-test-slave-vj6n krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266111, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:08:32.158932 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.153354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.13.8.65:43957)
0924 16:08:32.153545 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 16:08:32.153549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:08:32.153562 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:08:32.153648 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:08:32.153651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:08:32.153718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:08:32.153815 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:08:32.154196 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.154987 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.156232 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:08:32.156552 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:08:32.156651 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:08:32.157591 (+ 940us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:08:32.158254 (+ 663us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:08:32.158375 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 16:08:32.158721 (+ 346us) server_negotiation.cc:299] Negotiation successful
0924 16:08:32.158792 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260924 16:08:32.163779 24870 tablet.cc:2369] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:08:32.167060 25095 raft_consensus.cc:1261] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 1 FOLLOWER]: Refusing update from remote peer a1d76cbb30b445f89853d70c24d4268e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:08:32.168141 25175 consensus_queue.cc:1035] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [LEADER]: Connected to new peer: Peer: permanent_uuid: "157e611a4932460485c0ac85fca81a33" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:08:32.173982 24959 raft_consensus.cc:1261] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 1 FOLLOWER]: Refusing update from remote peer a1d76cbb30b445f89853d70c24d4268e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:08:32.175346 25175 consensus_queue.cc:1035] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a204237fdb4e4aedadf455fdcc7f3b66" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:08:32.190635 25179 mvcc.cc:205] Tried to move back new op lower bound from 7332929995427594240 to 7332929995233464320. Current Snapshot: MvccSnapshot[applied={T|T < 7332929995427594240}]
I20260924 16:08:32.201388 25182 mvcc.cc:205] Tried to move back new op lower bound from 7332929995427594240 to 7332929995233464320. Current Snapshot: MvccSnapshot[applied={T|T < 7332929995427594240}]
I20260924 16:08:32.221521 24655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:08:32.221732 24655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:08:32.224928 24655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e884d19634a44dbeae1e0fbd535c9192: Sending DeleteTablet for 3 replicas of tablet 4363ee27a991440097248550539ae9e5
I20260924 16:08:32.226972 24784 tablet_service.cc:1514] Processing DeleteTablet for tablet 4363ee27a991440097248550539ae9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:08:32 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:38772
I20260924 16:08:32.227133 25078 tablet_service.cc:1514] Processing DeleteTablet for tablet 4363ee27a991440097248550539ae9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:08:32 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:44210
I20260924 16:08:32.228317 24935 tablet_service.cc:1514] Processing DeleteTablet for tablet 4363ee27a991440097248550539ae9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:08:32 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:40994
I20260924 16:08:32.228356 25188 tablet_replica.cc:330] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e: stopping tablet replica
I20260924 16:08:32.228765 25188 raft_consensus.cc:2229] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:08:32.229236 25188 raft_consensus.cc:2258] T 4363ee27a991440097248550539ae9e5 P a1d76cbb30b445f89853d70c24d4268e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:08:32.229290 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 24737
I20260924 16:08:32.231865 25190 tablet_replica.cc:330] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: stopping tablet replica
I20260924 16:08:32.232193 25190 raft_consensus.cc:2229] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:08:32.232564 25190 raft_consensus.cc:2258] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:08:32.235527 25189 tablet_replica.cc:330] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: stopping tablet replica
I20260924 16:08:32.235836 25189 raft_consensus.cc:2229] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:08:32.236234 25189 raft_consensus.cc:2258] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:08:32.236349 25190 ts_tablet_manager.cc:1905] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:08:32.239756 25190 ts_tablet_manager.cc:1918] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:08:32.245903 25190 log.cc:1196] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/4363ee27a991440097248550539ae9e5
I20260924 16:08:32.247761 25189 ts_tablet_manager.cc:1905] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:08:32.250895 25189 ts_tablet_manager.cc:1918] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:08:32.251427 25190 ts_tablet_manager.cc:1939] T 4363ee27a991440097248550539ae9e5 P a204237fdb4e4aedadf455fdcc7f3b66: Deleting consensus metadata
I20260924 16:08:32.253007 24641 catalog_manager.cc:4841] TS a204237fdb4e4aedadf455fdcc7f3b66 (127.13.8.66:35287): tablet 4363ee27a991440097248550539ae9e5 (table test-table [id=f3fe858bd82d412e9f82d8f424299590]) successfully deleted
I20260924 16:08:32.253312 25189 log.cc:1196] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/4363ee27a991440097248550539ae9e5
I20260924 16:08:32.253738 25189 ts_tablet_manager.cc:1939] T 4363ee27a991440097248550539ae9e5 P 157e611a4932460485c0ac85fca81a33: Deleting consensus metadata
I20260924 16:08:32.255939 24642 catalog_manager.cc:4841] TS 157e611a4932460485c0ac85fca81a33 (127.13.8.67:36927): tablet 4363ee27a991440097248550539ae9e5 (table test-table [id=f3fe858bd82d412e9f82d8f424299590]) successfully deleted
W20260924 16:08:32.255967 24640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:43957 (error 108)
I20260924 16:08:32.256469 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 24877
I20260924 16:08:32.258181 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:08:32.257828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43957 (local address 127.0.0.1:38788)
0924 16:08:32.258001 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 16:08:32.258085 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:08:32.258411 24640 catalog_manager.cc:4584] TS a1d76cbb30b445f89853d70c24d4268e (127.13.8.65:43957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4363ee27a991440097248550539ae9e5: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43957: connect: Connection refused (error 111)
I20260924 16:08:32.266477 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 25013
I20260924 16:08:32.274775 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 24622
I20260924 16:08:32.284106 13345 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 16:08:32.628584 13345 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 16:08:32.628706 13345 mini_ranger.cc:66] Stopping Ranger...
I20260924 16:08:32.973520 13345 mini_ranger.cc:68] Stopped Ranger
2026-09-24 16:08:32.973 UTC [24204] LOG: received smart shutdown request
2026-09-24 16:08:32.978 UTC [24204] LOG: background worker "logical replication launcher" (PID 24213) exited with exit code 1
2026-09-24 16:08:32.978 UTC [24208] LOG: shutting down
2026-09-24 16:08:33.168 UTC [24204] LOG: database system is shut down
2026-09-24T16:08:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50646 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:08:33 dist-test-slave-vj6n krb5kdc[25201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:08:33 dist-test-slave-vj6n krb5kdc[25201](info): set up 2 sockets
Sep 24 16:08:33 dist-test-slave-vj6n krb5kdc[25201](info): commencing operation
krb5kdc: starting...
W20260924 16:08:34.394035 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:08:34 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:08:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:08:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangeradmin_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangeradmin_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangerlookup_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangerlookup_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
I20260924 16:08:34.564494 13345 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskf9hPi5/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/postgres -l logfile start
2026-09-24 16:08:37.040 UTC [25232] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 16:08:37.040 UTC [25232] LOG: listening on IPv4 address "127.13.8.84", port 38553
2026-09-24 16:08:37.043 UTC [25232] LOG: listening on Unix socket "/tmp/.s.PGSQL.38553"
2026-09-24 16:08:37.079 UTC [25235] LOG: database system was shut down at 2026-09-24 16:08:35 UTC
2026-09-24 16:08:37.083 UTC [25232] LOG: database system is ready to accept connections
I20260924 16:08:38.026969 13345 mini_postgres.cc:96] Postgres bound to 38553
2026-09-24 16:08:38.033 UTC [25243] FATAL: database "slave" does not exist
127.13.8.84:38553 - accepting connections
I20260924 16:08:38.033977 13345 mini_ranger.cc:161] Starting Ranger...
I20260924 16:08:38.049461 13345 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 16:08:38.538106 13345 mini_ranger.cc:87] Created ranger Postgres database
I20260924 16:08:38.538200 13345 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin
2026-09-24 16:08:38,815 [I] DB FLAVOR :POSTGRES
2026-09-24 16:08:38,816 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 16:08:38,816 [I] Checking connection..
2026-09-24 16:08:38,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 16:08:39,067 [I] Checking connection passed.
2026-09-24 16:08:39,067 [I] --------- Verifying version history table ---------
2026-09-24 16:08:39,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:08:39,331 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 16:08:39,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:08:39,591 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 16:08:39,591 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 16:08:39,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 16:08:39.820 UTC [25308] WARNING: there is no transaction in progress
2026-09-24 16:08:39,826 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 16:08:39,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 16:08:40,085 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 16:08:40,085 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 16:08:40,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 16:08:40,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 16:08:40,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vj6n.c.gcp-upstream.internal','N') ;"
2026-09-24 16:08:40,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 16:08:41,087 [I] Table x_portal_user does not exist in database ranger
2026-09-24 16:08:41,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 16:08:41,336 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 16:08:41,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 16:08:41,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 16:08:41,820 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 16:08:41,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 16:08:42.788 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.805 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.821 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.836 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.851 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.870 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.925 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.932 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.947 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.958 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.968 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.981 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.991 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:42.997 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:43.003 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:43.010 UTC [25419] WARNING: there is no transaction in progress
2026-09-24 16:08:43,652 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 16:08:43,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 16:08:43,907 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 16:08:43,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 16:08:44,161 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 16:08:44,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 16:08:44,424 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 16:08:44,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 16:08:44,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 16:08:44,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vj6n.c.gcp-upstream.internal';"
2026-09-24 16:08:45,150 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 16:08:45,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 16:08:45,383 [I] DB_PATCHES have already been applied
I20260924 16:08:45.388752 13345 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskf9hPi5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 16:08:45.388823 13345 mini_ranger.cc:193] Using host: 127.13.8.84
I20260924 16:08:45.392220 13345 mini_ranger.cc:227] Ranger admin URL: http://127.13.8.84:45193
Sep 24, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 4:08:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 4:08:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 16:08:46.515482 13345 mini_ranger.cc:160] Time spent starting Ranger: real 8.482s user 0.003s sys 0.006s
Sep 24, 2026 4:09:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 4:09:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 24, 2026 4:09:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 4:09:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 4:09:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 16:09:08.239159 13345 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangerkms_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/rangerkms_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/HTTP_127.13.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 16:09:08 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 16:09:08.325457 13345 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 16:09:08.335852 13345 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 16:09:09.226269 13345 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 16:09:09.226341 13345 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms
I20260924 16:09:09.226354 13345 mini_ranger_kms.cc:227] Using postgres at 127.13.8.84:38553
2026-09-24 16:09:09,399 [I] DB FLAVOR :POSTGRES
2026-09-24 16:09:09,399 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 16:09:09,399 [I] Checking connection
2026-09-24 16:09:09,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 16:09:09,632 [I] connection success
2026-09-24 16:09:09,632 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 16:09:09,632 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 16:09:09,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 16:09:09,874 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 16:09:09,874 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 16:09:09,874 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 16:09:09,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.8.84:38553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 16:09:10,151 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 16:09:10.615384 13345 mini_ranger_kms.cc:310] Created kms service
I20260924 16:09:10.810191 13345 mini_ranger_kms.cc:326] Created kudu user
I20260924 16:09:10.852087 13345 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 16:09:10 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:11.247133 13345 mini_ranger_kms.cc:384] Added ranger policy
I20260924 16:09:11.247288 13345 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms:/tmp/dist-test-taskf9hPi5/build/debug/bin/postgresql.jar:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskf9hPi5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 16:09:11.247309 13345 mini_ranger_kms.cc:240] Using host: 127.13.8.84
I20260924 16:09:11.251612 13345 mini_ranger_kms.cc:276] Ranger KMS PID: 25602
I20260924 16:09:11.251690 13345 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.8.84:47959
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 4:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 16:09:13.485666 13345 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.160s user 0.001s sys 0.155s
I20260924 16:09:13.485810 13345 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 16:09:13.485888 13345 mini_ranger_kms.cc:401] 127.13.8.84:47959/kms/v1/keys
Sep 24 16:09:13 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790266153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 16:09:14 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
Sep 24 16:09:14 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.8.84@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
Sep 24 16:09:16 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:09:16.818186 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44159
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:44159
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:16.929611 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:16.929909 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:16.929989 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:16.934005 25654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:16.934105 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:16.934141 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:16.934180 25654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:16.934209 25654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:16.939795 25654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.8.84:47959/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:44159
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:16.940927 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:16.942019 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:16.950750 25661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:16.950765 25660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:16.950832 25654 server_base.cc:1034] running on GCE node
W20260924 16:09:16.950765 25663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:16.951510 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:16.952638 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:16.953832 25654 hybrid_clock.cc:648] HybridClock initialized: now 1790266156953814 us; error 37 us; skew 500 ppm
Sep 24 16:09:16 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266156, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:16.956724 25654 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:16.957705 25654 webserver.cc:460] Webserver started at http://127.13.8.126:37983/ using document root <none> and password file <none>
I20260924 16:09:16.958321 25654 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:16.958405 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:16.958614 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:09:16 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266156, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:17.019771 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "204c18a1d9ef458bbbd86a6e51f7186d"
format_stamp: "Formatted at 2026-09-24 16:09:17 on dist-test-slave-vj6n"
server_key: "900fc1e0872e1a49a3f404c47e39d81a"
server_key_iv: "0780930250abf7f46eab29c05aeff18c"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:17.020421 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "204c18a1d9ef458bbbd86a6e51f7186d"
format_stamp: "Formatted at 2026-09-24 16:09:17 on dist-test-slave-vj6n"
server_key: "900fc1e0872e1a49a3f404c47e39d81a"
server_key_iv: "0780930250abf7f46eab29c05aeff18c"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:17.024567 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:09:17.042410 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:17.043869 25654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:17.043993 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "204c18a1d9ef458bbbd86a6e51f7186d"
format_stamp: "Formatted at 2026-09-24 16:09:17 on dist-test-slave-vj6n"
server_key: "900fc1e0872e1a49a3f404c47e39d81a"
server_key_iv: "0780930250abf7f46eab29c05aeff18c"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:17.044164 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:17.064278 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:17.067440 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:17.067667 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:17.075830 25654 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:44159
I20260924 16:09:17.075834 25722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:44159 every 8 connection(s)
I20260924 16:09:17.076885 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:17.080794 25723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:17.085737 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 25654
I20260924 16:09:17.085891 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:09:17.086199 13345 external_mini_cluster.cc:1428] Setting key ba25ebcaad04306389de2eee5413f230
I20260924 16:09:17.088760 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Bootstrap starting.
I20260924 16:09:17.091794 25723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:17.092756 25723 log.cc:824] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:17.095019 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: No bootstrap required, opened a new log
Sep 24 16:09:17 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:17.099246 25723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:17.099570 25723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:17.099664 25723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204c18a1d9ef458bbbd86a6e51f7186d, State: Initialized, Role: FOLLOWER
I20260924 16:09:17.100181 25723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:17.100347 25723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:17.100477 25723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:17.100602 25723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:17.101764 25723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:17.102136 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:17.087806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.13.8.126:44159)
0924 16:09:17.088250 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:17.088281 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:17.088325 (+ 44us) server_negotiation.cc:407] Connection header received
0924 16:09:17.089019 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:17.089039 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:17.089369 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:17.089707 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:17.090584 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:17.091568 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:17.092552 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:17.093097 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:17.093271 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:17.096458 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:17.096466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:17.096485 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:17.096519 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:17.098554 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:17.099078 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:17.099085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:17.099091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:17.099220 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:17.099539 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:17.099542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:17.099544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:17.100043 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:17.100237 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:17.100871 (+ 634us) server_negotiation.cc:299] Negotiation successful
0924 16:09:17.101160 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":139,"threads_started":1}
I20260924 16:09:17.102123 25723 leader_election.cc:302] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 204c18a1d9ef458bbbd86a6e51f7186d; no voters:
I20260924 16:09:17.103345 25723 leader_election.cc:288] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:09:17.103469 25728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:17.103729 25728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 LEADER]: Becoming Leader. State: Replica: 204c18a1d9ef458bbbd86a6e51f7186d, State: Running, Role: LEADER
I20260924 16:09:17.104076 25728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:17.104979 25723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:17.105324 25728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } } }
I20260924 16:09:17.105463 25728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:17.105638 25728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 204c18a1d9ef458bbbd86a6e51f7186d. Latest consensus state: current_term: 1 leader_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } } }
I20260924 16:09:17.105705 25728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:17.106478 25732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:17.109691 25732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:17.110092 25723 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 16:09:17.120231 25732 catalog_manager.cc:1347] Generated new cluster ID: c9dceb7ee8074cbd9679b1acdb57f536
I20260924 16:09:17.120383 25732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:17.137915 25732 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:09:17.139358 25732 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:17.149821 25732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Generated new TSK 0
I20260924 16:09:17.150944 25732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:09:17.969978 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:18.140314 25769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:18.140831 25769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:18.140964 25769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:18.147058 25769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:18.147140 25769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:18.147233 25769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:18.155254 25769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:18.156596 25769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:18.157800 25769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:18.170851 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:18.171217 25777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:18.172350 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:18.172513 25769 server_base.cc:1034] running on GCE node
I20260924 16:09:18.173168 25769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:18.173977 25769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:18.175166 25769 hybrid_clock.cc:648] HybridClock initialized: now 1790266158175131 us; error 55 us; skew 500 ppm
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:18.178722 25769 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:18.179934 25769 webserver.cc:460] Webserver started at http://127.13.8.65:42309/ using document root <none> and password file <none>
I20260924 16:09:18.180639 25769 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:18.180801 25769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:18.181124 25769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:18.203290 25769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "f9dd0af246184f39984f3ede61402269"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "35a4b427270689d30d1ecd96d4f83bda"
server_key_iv: "bba5f496d95c2138c81f0713e8c457ed"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.204022 25769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "f9dd0af246184f39984f3ede61402269"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "35a4b427270689d30d1ecd96d4f83bda"
server_key_iv: "bba5f496d95c2138c81f0713e8c457ed"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.209511 25769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 16:09:18.226616 25786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.228356 25769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 16:09:18.228521 25769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "f9dd0af246184f39984f3ede61402269"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "35a4b427270689d30d1ecd96d4f83bda"
server_key_iv: "bba5f496d95c2138c81f0713e8c457ed"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.228680 25769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:18.243816 25769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:18.247677 25769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:18.247920 25769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:18.248668 25769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:18.249785 25769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:18.249899 25769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.250029 25769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:18.250120 25769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.263839 25769 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:38695
I20260924 16:09:18.265308 25769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:18.266367 25901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:38695 every 8 connection(s)
I20260924 16:09:18.274816 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 25769
I20260924 16:09:18.274969 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:09:18.275272 13345 external_mini_cluster.cc:1428] Setting key 1f8e9e0d0d2ca3f92734e7bcfed211f0
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:18.284294 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:18.268788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:43871 (local address 127.13.8.126:44159)
0924 16:09:18.269053 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:18.269058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:18.269945 (+ 887us) server_negotiation.cc:407] Connection header received
0924 16:09:18.271143 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:18.271147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:18.271214 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:18.271282 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:18.273034 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.273926 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.275903 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.276273 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.276351 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:18.279613 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:18.279617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:18.279620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:18.279650 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:18.281590 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:18.282298 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:18.282301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:18.282303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:18.282399 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:18.282834 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:18.282837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:18.282839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:18.283092 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:18.283194 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:18.283957 (+ 763us) server_negotiation.cc:299] Negotiation successful
0924 16:09:18.284100 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260924 16:09:18.285724 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:18.269292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.65:43871)
0924 16:09:18.269785 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:18.269839 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:18.270830 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:18.271740 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:18.271749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:18.272128 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:18.272850 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:18.272872 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.274098 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.274101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:18.275757 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:18.275769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.276554 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.276558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:18.276785 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:18.277522 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:18.277550 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:18.279407 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:18.281739 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:18.281745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:18.281763 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:18.282149 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:18.282509 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:18.282512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:18.282514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:18.282712 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:18.283269 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:18.283275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:18.283643 (+ 368us) client_negotiation.cc:769] Sending connection context
0924 16:09:18.284577 (+ 934us) client_negotiation.cc:240] Negotiation successful
0924 16:09:18.284921 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":147,"threads_started":1}
I20260924 16:09:18.287444 25902 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:18.287715 25902 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:18.288193 25902 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:18.289795 25687 ts_manager.cc:194] Registered new tserver with Master: f9dd0af246184f39984f3ede61402269 (127.13.8.65:38695)
I20260924 16:09:18.291500 25687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:43871
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:09:18.339334 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 16:09:18.503909 25910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:18.504402 25910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:18.504518 25910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:18.510866 25910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:18.511008 25910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:18.511155 25910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:18.519218 25910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:18.520931 25910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:18.522192 25910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:18.532415 25916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:18.532415 25917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:18.533033 25919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:18.533566 25910 server_base.cc:1034] running on GCE node
I20260924 16:09:18.534107 25910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:18.534888 25910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:18.542347 25910 hybrid_clock.cc:648] HybridClock initialized: now 1790266158542297 us; error 60 us; skew 500 ppm
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:18.545564 25910 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:18.546859 25910 webserver.cc:460] Webserver started at http://127.13.8.66:46325/ using document root <none> and password file <none>
I20260924 16:09:18.547638 25910 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:18.547725 25910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:18.547957 25910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:18.572801 25910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "001301b466924d609503bfd4d61a2dc0"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "c6c0ba51e76074779a5f0b38a5374edb"
server_key_iv: "1f859357cc01ab480d40daf98ba5c6a2"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.574143 25910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "001301b466924d609503bfd4d61a2dc0"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "c6c0ba51e76074779a5f0b38a5374edb"
server_key_iv: "1f859357cc01ab480d40daf98ba5c6a2"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.583659 25910 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.003s
I20260924 16:09:18.604591 25926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.606698 25910 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260924 16:09:18.606841 25910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "001301b466924d609503bfd4d61a2dc0"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "c6c0ba51e76074779a5f0b38a5374edb"
server_key_iv: "1f859357cc01ab480d40daf98ba5c6a2"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.607031 25910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:18.633909 25910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:18.637327 25910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:18.637576 25910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:18.638389 25910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:18.639355 25910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:18.639438 25910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.639559 25910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:18.639618 25910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.652415 25910 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39303
I20260924 16:09:18.652982 26041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39303 every 8 connection(s)
I20260924 16:09:18.653837 25910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:18.656351 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 25910
I20260924 16:09:18.656491 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:09:18.656800 13345 external_mini_cluster.cc:1428] Setting key ecea907bcd4a5e5db07521128f1d64f1
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:18.696599 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:18.670163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:46559 (local address 127.13.8.126:44159)
0924 16:09:18.670338 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:18.670343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:18.680052 (+ 9709us) server_negotiation.cc:407] Connection header received
0924 16:09:18.683908 (+ 3856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:18.683913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:18.683993 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:18.684052 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:18.685980 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.686831 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.688527 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.688849 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.688926 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:18.692078 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:18.692082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:18.692085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:18.692115 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:18.693914 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:18.694640 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:18.694644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:18.694646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:18.694737 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:18.695169 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:18.695172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:18.695174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:18.695434 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:18.695519 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:18.696280 (+ 761us) server_negotiation.cc:299] Negotiation successful
0924 16:09:18.696423 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 16:09:18.698606 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:18.668928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.66:46559)
0924 16:09:18.679829 (+ 10901us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:18.679914 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:18.683587 (+ 3673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:18.684559 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:18.684571 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:18.684990 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:18.685750 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:18.685771 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.687010 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.687014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:18.688377 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:18.688388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:18.689153 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:18.689157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:18.689357 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:18.690060 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:18.690082 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:18.691856 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:18.694069 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:18.694076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:18.694096 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:18.694510 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:18.694853 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:18.694855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:18.694857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:18.695053 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:18.695589 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:18.695596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:18.695947 (+ 351us) client_negotiation.cc:769] Sending connection context
0924 16:09:18.697091 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0924 16:09:18.697462 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10583,"thread_start_us":170,"threads_started":1}
I20260924 16:09:18.700214 26042 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:18.700460 26042 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:18.701076 26042 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:18.702402 25687 ts_manager.cc:194] Registered new tserver with Master: 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303)
I20260924 16:09:18.703158 25687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:46559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:09:18.710537 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:18.876539 26047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:18.877389 26047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:18.877547 26047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:18.883741 26047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:18.883862 26047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:18.883993 26047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:18.891885 26047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:18.893555 26047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:18.895002 26047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:18.908834 26053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:18.910175 26047 server_base.cc:1034] running on GCE node
W20260924 16:09:18.914371 26052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:18.914655 26055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:18.915197 26047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:18.916038 26047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:18.917188 26047 hybrid_clock.cc:648] HybridClock initialized: now 1790266158917113 us; error 93 us; skew 500 ppm
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:18.921193 26047 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:18.922390 26047 webserver.cc:460] Webserver started at http://127.13.8.67:41267/ using document root <none> and password file <none>
I20260924 16:09:18.923319 26047 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:18.923448 26047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:18.923748 26047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 16:09:18 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:18.943318 26047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "f531d27908364208af80dfeb7c490867"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "4500d740724ac0678ac072a186c07ce1"
server_key_iv: "67a8c48abc58cf99c4b657e106eba956"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.944152 26047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "f531d27908364208af80dfeb7c490867"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "4500d740724ac0678ac072a186c07ce1"
server_key_iv: "67a8c48abc58cf99c4b657e106eba956"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.959604 26047 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260924 16:09:18.981302 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:18.983533 26047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:09:18.983692 26047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "f531d27908364208af80dfeb7c490867"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "4500d740724ac0678ac072a186c07ce1"
server_key_iv: "67a8c48abc58cf99c4b657e106eba956"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:18.983855 26047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:19.020442 26047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:19.024320 26047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:19.024585 26047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:19.025377 26047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:19.026511 26047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:19.026607 26047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.026697 26047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:19.026753 26047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.037408 26047 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:39375
I20260924 16:09:19.038738 26047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:09:19.047363 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26047
I20260924 16:09:19.047497 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:09:19.047807 13345 external_mini_cluster.cc:1428] Setting key 6f2afd6a5860ea4da0ea588bacea56cb
I20260924 16:09:19.049288 26176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:39375 every 8 connection(s)
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266158, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:19.097177 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.053149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:33005 (local address 127.13.8.126:44159)
0924 16:09:19.053271 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:19.053275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:19.062458 (+ 9183us) server_negotiation.cc:407] Connection header received
0924 16:09:19.063792 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:19.063795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:19.063847 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:19.063895 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:19.071055 (+ 7160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.071824 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.073662 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.073985 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.074058 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.092565 (+ 18507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:19.092570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:19.092585 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:19.092628 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:19.094450 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.095155 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.095159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.095161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.095246 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.095699 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.095704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.095705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.095943 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:19.096030 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:19.096871 (+ 841us) server_negotiation.cc:299] Negotiation successful
0924 16:09:19.097012 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 16:09:19.098923 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.052043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.67:33005)
0924 16:09:19.062261 (+ 10218us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:19.062353 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:19.063535 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:19.064450 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:19.064461 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:19.064862 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:19.065592 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.065611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.072043 (+ 6432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.072047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.073510 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.073523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.074315 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.074319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.074536 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.090460 (+ 15924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:19.090489 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:19.092303 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:19.094630 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.094638 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.094661 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.095033 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.095355 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.095357 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.095359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.095563 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.096103 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:19.096110 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:19.096550 (+ 440us) client_negotiation.cc:769] Sending connection context
0924 16:09:19.097410 (+ 860us) client_negotiation.cc:240] Negotiation successful
0924 16:09:19.097794 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9855,"thread_start_us":154,"threads_started":1}
I20260924 16:09:19.100241 26177 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:19.100505 26177 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.101087 26177 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:19.102265 25687 ts_manager.cc:194] Registered new tserver with Master: f531d27908364208af80dfeb7c490867 (127.13.8.67:39375)
I20260924 16:09:19.103050 25687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:33005
I20260924 16:09:19.108549 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:09:19.109375 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 25769
I20260924 16:09:19.121479 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 25910
I20260924 16:09:19.130971 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26047
I20260924 16:09:19.139355 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 25654
I20260924 16:09:19.147846 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44159
--webserver_interface=127.13.8.126
--webserver_port=37983
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:44159
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:19.261680 26180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:19.261946 26180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:19.261986 26180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:19.266311 26180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:19.266366 26180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:19.266384 26180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:19.266428 26180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:19.266461 26180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:19.271868 26180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:44159
--ranger_config_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=37983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:19.272919 26180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:19.273965 26180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:19.282760 26188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:19.282848 26185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:19.282915 26180 server_base.cc:1034] running on GCE node
W20260924 16:09:19.282843 26186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:19.283694 26180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:19.284932 26180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:19.286108 26180 hybrid_clock.cc:648] HybridClock initialized: now 1790266159286089 us; error 34 us; skew 500 ppm
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:19.288987 26180 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:19.289877 26180 webserver.cc:460] Webserver started at http://127.13.8.126:37983/ using document root <none> and password file <none>
I20260924 16:09:19.290501 26180 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:19.290567 26180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:19.354562 26180 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:09:19.357647 26196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.359120 26180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:09:19.359292 26180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "204c18a1d9ef458bbbd86a6e51f7186d"
format_stamp: "Formatted at 2026-09-24 16:09:17 on dist-test-slave-vj6n"
server_key: "900fc1e0872e1a49a3f404c47e39d81a"
server_key_iv: "0780930250abf7f46eab29c05aeff18c"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:19.359874 26180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:19.375192 26180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:19.378086 26180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:19.378562 26180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:19.385864 26180 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:44159
I20260924 16:09:19.385886 26248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:44159 every 8 connection(s)
I20260924 16:09:19.387028 26180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:19.391397 26249 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:09:19.393074 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Bootstrap starting.
I20260924 16:09:19.396034 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26180
I20260924 16:09:19.396521 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:38695
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:44159
--webserver_port=42309
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:19.410018 26249 log.cc:824] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:19.417006 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:19.417452 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Bootstrap complete.
I20260924 16:09:19.421887 26249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:19.422340 26249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204c18a1d9ef458bbbd86a6e51f7186d, State: Initialized, Role: FOLLOWER
I20260924 16:09:19.422912 26249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:19.423033 26249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:19.423130 26249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:19.423225 26249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:09:19.425061 26249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:19.425397 26249 leader_election.cc:302] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 204c18a1d9ef458bbbd86a6e51f7186d; no voters:
I20260924 16:09:19.425731 26249 leader_election.cc:288] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 16:09:19.425814 26253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 2 FOLLOWER]: Leader election won for term 2
I20260924 16:09:19.426085 26253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [term 2 LEADER]: Becoming Leader. State: Replica: 204c18a1d9ef458bbbd86a6e51f7186d, State: Running, Role: LEADER
I20260924 16:09:19.426445 26253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } }
I20260924 16:09:19.427011 26249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:19.427425 26253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 204c18a1d9ef458bbbd86a6e51f7186d. Latest consensus state: current_term: 2 leader_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } } }
I20260924 16:09:19.427631 26253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:19.427858 26254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204c18a1d9ef458bbbd86a6e51f7186d" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 44159 } } }
I20260924 16:09:19.427968 26254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:19.428303 26257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:19.431737 26257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:19.432237 26249 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 16:09:19.432443 26257 catalog_manager.cc:1259] Loaded cluster ID: c9dceb7ee8074cbd9679b1acdb57f536
I20260924 16:09:19.432503 26257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:19.437827 26257 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:19.439155 26257 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Loaded TSK: 0
I20260924 16:09:19.440526 26257 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:09:19.522873 26251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:19.523226 26251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:19.523344 26251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:19.528187 26251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:19.528285 26251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:19.528411 26251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:19.534435 26251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:38695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=42309
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:19.535943 26251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:19.537464 26251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:19.550580 26288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:19.550689 26287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:19.550957 26290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:19.551206 26251 server_base.cc:1034] running on GCE node
I20260924 16:09:19.551640 26251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:19.552364 26251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:19.553563 26251 hybrid_clock.cc:648] HybridClock initialized: now 1790266159553521 us; error 51 us; skew 500 ppm
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:19.556465 26251 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:19.557495 26251 webserver.cc:460] Webserver started at http://127.13.8.65:42309/ using document root <none> and password file <none>
I20260924 16:09:19.558255 26251 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:19.558344 26251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:19.578104 26251 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:19.582674 26298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.584304 26251 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260924 16:09:19.584446 26251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "f9dd0af246184f39984f3ede61402269"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "35a4b427270689d30d1ecd96d4f83bda"
server_key_iv: "bba5f496d95c2138c81f0713e8c457ed"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:19.585170 26251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:19.603574 26251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:19.607102 26251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:19.607324 26251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:19.608107 26251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:19.609186 26251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:19.609259 26251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.609323 26251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:19.609349 26251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.621189 26251 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:38695
I20260924 16:09:19.621210 26413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:38695 every 8 connection(s)
I20260924 16:09:19.622644 26251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:19.623755 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26251
I20260924 16:09:19.624208 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:39303
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:44159
--webserver_port=46325
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:19.655261 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.627698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.65:33935)
0924 16:09:19.637162 (+ 9464us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:19.637239 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:19.638458 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:19.640845 (+ 2387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:19.640855 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:19.641212 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:19.641833 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.641852 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.643883 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.643887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.645311 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.645322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.646638 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.646642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.646834 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.647546 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:19.647570 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:19.649240 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:19.651773 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.651779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.651794 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.652132 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.652522 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.652525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.652527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.652688 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.653454 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:19.653460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:19.653748 (+ 288us) client_negotiation.cc:769] Sending connection context
0924 16:09:19.653954 (+ 206us) client_negotiation.cc:240] Negotiation successful
0924 16:09:19.654306 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9142,"thread_start_us":157,"threads_started":1}
I20260924 16:09:19.657040 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.627617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:33935 (local address 127.13.8.126:44159)
0924 16:09:19.639028 (+ 11411us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:19.639051 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:19.639097 (+ 46us) server_negotiation.cc:407] Connection header received
0924 16:09:19.639403 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:19.639417 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:19.639956 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:19.640342 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:19.642045 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.643722 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.645472 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.646206 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.646408 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.649475 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:19.649483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:19.649501 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:19.649536 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:19.651634 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.652271 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.652277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.652282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.652413 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.652796 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.652799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.652801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.653262 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:19.653374 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:19.655827 (+ 2453us) server_negotiation.cc:299] Negotiation successful
0924 16:09:19.656129 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11208,"thread_start_us":126,"threads_started":1}
I20260924 16:09:19.660343 26414 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:19.660593 26414 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.661157 26414 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:19.663694 26414 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:44159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 16:09:19.663834 26414 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.664017 26414 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:19.664951 26414 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.665126 26414 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:19.665933 26414 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:09:19.774201 26416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:19.774622 26416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:19.774689 26416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:19.780825 26416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:19.780907 26416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:19.780994 26416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:19.787307 26416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:39303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=46325
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:19.788502 26416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:19.789676 26416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:19.801074 26424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:19.801054 26423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:19.801658 26426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:19.801780 26416 server_base.cc:1034] running on GCE node
I20260924 16:09:19.802256 26416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:19.803033 26416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:19.804246 26416 hybrid_clock.cc:648] HybridClock initialized: now 1790266159804218 us; error 36 us; skew 500 ppm
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:19.807703 26416 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:19.808629 26416 webserver.cc:460] Webserver started at http://127.13.8.66:46325/ using document root <none> and password file <none>
I20260924 16:09:19.809386 26416 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:19.809466 26416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:19.829129 26416 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:09:19.833629 26434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.835240 26416 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260924 16:09:19.835403 26416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "001301b466924d609503bfd4d61a2dc0"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "c6c0ba51e76074779a5f0b38a5374edb"
server_key_iv: "1f859357cc01ab480d40daf98ba5c6a2"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:19.836040 26416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:19.851908 26416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:19.855376 26416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:19.855584 26416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:19.856228 26416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:19.857141 26416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:19.857193 26416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.857265 26416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:19.857307 26416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:19.867817 26416 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39303
I20260924 16:09:19.867986 26548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39303 every 8 connection(s)
I20260924 16:09:19.868960 26416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:19.872233 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26416
I20260924 16:09:19.872721 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.8.84:47959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:39375
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:44159
--webserver_port=41267
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:09:19 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266159, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:19.886672 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.871002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:43523 (local address 127.13.8.126:44159)
0924 16:09:19.871175 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:19.871180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:19.872075 (+ 895us) server_negotiation.cc:407] Connection header received
0924 16:09:19.873094 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:19.873097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:19.873160 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:19.873289 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:19.874840 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.875688 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.877597 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.878062 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.878161 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.881740 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:19.881746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:19.881752 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:19.881790 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:19.883728 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.884456 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.884463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.884467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.884572 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:19.885096 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:19.885103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:19.885107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:19.885386 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:19.885486 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:19.886338 (+ 852us) server_negotiation.cc:299] Negotiation successful
0924 16:09:19.886495 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:09:19.888327 26551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:19.871406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.66:43523)
0924 16:09:19.871920 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:19.871974 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:19.872860 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:19.873496 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:19.873504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:19.873841 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:19.874638 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.874659 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.875845 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.875852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.877431 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:19.877447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:19.878458 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:19.878465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:19.878714 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:19.879379 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:19.879418 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:19.881513 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:19.883879 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.883889 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.883911 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.884307 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.884709 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:19.884714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:19.884719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:19.884965 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:19.885584 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:19.885595 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:19.886007 (+ 412us) client_negotiation.cc:769] Sending connection context
0924 16:09:19.886861 (+ 854us) client_negotiation.cc:240] Negotiation successful
0924 16:09:19.887240 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":153,"threads_started":1}
I20260924 16:09:19.889598 26549 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:19.889851 26549 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.890508 26549 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:19.892309 26549 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:44159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 16:09:19.892447 26549 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.892637 26549 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:19.893443 26549 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:19.893611 26549 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:19.894357 26549 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:09:20.018170 26552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:20.018563 26552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:20.018651 26552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:20.024576 26552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:20.024670 26552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:20.024771 26552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:20.030848 26552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.8.84:47959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:39375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=41267
--tserver_master_addrs=127.13.8.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:20.032136 26552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:20.033346 26552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:20.045974 26559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:20.045982 26561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:20.047307 26558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:20.047526 26552 server_base.cc:1034] running on GCE node
I20260924 16:09:20.048146 26552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:20.048882 26552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:20.050098 26552 hybrid_clock.cc:648] HybridClock initialized: now 1790266160050065 us; error 48 us; skew 500 ppm
Sep 24 16:09:20 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:20.053231 26552 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:20.054509 26552 webserver.cc:460] Webserver started at http://127.13.8.67:41267/ using document root <none> and password file <none>
I20260924 16:09:20.055238 26552 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:20.055347 26552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 16:09:20 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:20.078760 26552 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:09:20.081929 26569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:20.083366 26552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:09:20.083474 26552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "f531d27908364208af80dfeb7c490867"
format_stamp: "Formatted at 2026-09-24 16:09:18 on dist-test-slave-vj6n"
server_key: "4500d740724ac0678ac072a186c07ce1"
server_key_iv: "67a8c48abc58cf99c4b657e106eba956"
server_key_version: "kuduclusterkey@0"
I20260924 16:09:20.084049 26552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:20.094636 26552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:20.097718 26552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:20.097937 26552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:20.098665 26552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:20.099707 26552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:20.099773 26552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:20.099871 26552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:20.099925 26552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:20.113438 26552 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:39375
I20260924 16:09:20.113623 26683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:39375 every 8 connection(s)
I20260924 16:09:20.114684 26552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:09:20.121760 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26552
Sep 24 16:09:20 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:20.137639 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:20.116647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:45669 (local address 127.13.8.126:44159)
0924 16:09:20.116783 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:20.116787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:20.117775 (+ 988us) server_negotiation.cc:407] Connection header received
0924 16:09:20.118893 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:20.118896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:20.118957 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:20.119067 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:20.120423 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.120967 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.127768 (+ 6801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.128168 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.128272 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:20.131860 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:20.131864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:20.131866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:20.131891 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:20.133702 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:20.135126 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:20.135131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:20.135133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:20.135228 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:20.135726 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:20.135729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:20.135731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:20.135971 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:20.136068 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:20.137233 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0924 16:09:20.137441 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":13184}
I20260924 16:09:20.138517 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:20.117106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:44159 (local address 127.13.8.67:45669)
0924 16:09:20.117617 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:20.117669 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:20.118662 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:20.119271 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:20.119278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:20.119551 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:20.120231 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:20.120252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.121123 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.121126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:20.127575 (+ 6449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:20.127590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.128561 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.128565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:20.128795 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:20.129767 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:20.129792 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:20.131673 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:20.133815 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:20.133821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:20.133840 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:20.134240 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:20.135359 (+ 1119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:20.135365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:20.135369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:20.135601 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:20.136174 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:20.136182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:20.136573 (+ 391us) client_negotiation.cc:769] Sending connection context
0924 16:09:20.136848 (+ 275us) client_negotiation.cc:240] Negotiation successful
0924 16:09:20.137254 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":153,"threads_started":1}
I20260924 16:09:20.139976 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:20.122901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.13.8.126:44159)
0924 16:09:20.129284 (+ 6383us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:20.129289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:20.129304 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:09:20.129345 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:20.129348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:20.129400 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:20.129517 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:20.130664 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.132196 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.133657 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:20.134002 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:20.134105 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:20.135877 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:20.135881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:20.135883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:20.135963 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:20.137911 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:20.138441 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:20.138444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:20.138446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:20.138533 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:20.138917 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:20.138920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:20.138922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:20.139132 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:20.139338 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:20.139593 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 16:09:20.139796 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6291,"spinlock_wait_cycles":4224,"thread_start_us":89,"threads_started":1}
I20260924 16:09:20.140211 26684 heartbeater.cc:347] Connected to a master server at 127.13.8.126:44159
I20260924 16:09:20.140473 26684 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:20.141129 26684 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:20.142882 26684 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:44159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 16:09:20.143043 26684 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:20.143229 26684 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:20.144098 26684 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:20.144287 26684 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
W20260924 16:09:20.145102 26684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 16:09:20 dist-test-slave-vj6n krb5kdc[25201](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:20.666188 26414 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:20.666548 26414 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:20.667935 26213 ts_manager.cc:194] Registered new tserver with Master: f9dd0af246184f39984f3ede61402269 (127.13.8.65:38695)
I20260924 16:09:20.669773 26213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:33935
I20260924 16:09:20.894582 26549 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:20.894889 26549 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:20.895671 26213 ts_manager.cc:194] Registered new tserver with Master: 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303)
I20260924 16:09:20.896507 26213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:43523
Sep 24 16:09:20 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:21.145341 26684 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:21.145627 26684 heartbeater.cc:511] Master 127.13.8.126:44159 requested a full tablet report, sending...
I20260924 16:09:21.146512 26213 ts_manager.cc:194] Registered new tserver with Master: f531d27908364208af80dfeb7c490867 (127.13.8.67:39375)
I20260924 16:09:21.147316 26213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:45669
I20260924 16:09:21.156903 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:09:21.169842 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.158834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.13.8.126:44159)
0924 16:09:21.159221 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.159226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.159552 (+ 326us) server_negotiation.cc:407] Connection header received
0924 16:09:21.159711 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.159714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.159770 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.159851 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:21.160863 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.161730 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.163011 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.163432 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.163544 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.165544 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:21.165547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:21.165550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:21.165580 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:21.167497 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:21.168127 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:21.168131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:21.168132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:21.168234 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:21.168586 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:21.168592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:21.168596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:21.168938 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:21.169096 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.169354 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.169492 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":95,"threads_started":1}
I20260924 16:09:21.174397 26213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:09:21.179390 26213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 24 16:09:21 dist-test-slave-vj6n krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266160, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for HTTP/127.13.8.84@KRBTEST.COM
I20260924 16:09:21.212200 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.196273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:38695 (local address 127.0.0.1:42736)
0924 16:09:21.202292 (+ 6019us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.202358 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.202511 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.203785 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.203793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.203818 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.204178 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.204191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.207285 (+ 3094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.207289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.209516 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.209528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.211273 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.211277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.211398 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.211451 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.211725 (+ 274us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.211871 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5580,"thread_start_us":91,"threads_started":1}
I20260924 16:09:21.213723 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.196387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.13.8.65:38695)
0924 16:09:21.202606 (+ 6219us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.202614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.202642 (+ 28us) server_negotiation.cc:407] Connection header received
0924 16:09:21.203144 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.203152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.203402 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.203580 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.204472 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.207162 (+ 2690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.209665 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.210774 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.210878 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.213254 (+ 2376us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.213312 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.213580 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6077,"thread_start_us":79,"threads_started":1}
I20260924 16:09:21.217306 26340 tablet_service.cc:1467] Processing CreateTablet for tablet 6fafd9dcc09545aca7993e5429832244 (DEFAULT_TABLE table=test-table [id=32546ebdee5647ba84434280ddda8a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:21.218549 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fafd9dcc09545aca7993e5429832244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:21.220285 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.207499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39303 (local address 127.0.0.1:57318)
0924 16:09:21.212759 (+ 5260us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.212773 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.212840 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.215956 (+ 3116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.215960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.215979 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.216070 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.216076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.217717 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.217721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.218756 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.218768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.219905 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.219909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.219999 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.220027 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.220079 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.220148 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5199,"thread_start_us":109,"threads_started":1}
I20260924 16:09:21.222056 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.207691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.13.8.66:39303)
0924 16:09:21.215363 (+ 7672us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.215370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.215448 (+ 78us) server_negotiation.cc:407] Connection header received
0924 16:09:21.215537 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.215542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.215786 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.216287 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.216453 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.217583 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.218908 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.219763 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.221607 (+ 1844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.221770 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.221820 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.221916 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7541,"thread_start_us":98,"threads_started":1}
I20260924 16:09:21.225216 26482 tablet_service.cc:1467] Processing CreateTablet for tablet 6fafd9dcc09545aca7993e5429832244 (DEFAULT_TABLE table=test-table [id=32546ebdee5647ba84434280ddda8a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:21.226429 26482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fafd9dcc09545aca7993e5429832244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:21.226572 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.208201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:39375 (local address 127.0.0.1:51350)
0924 16:09:21.220457 (+ 12256us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.220471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.220565 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.221300 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.221304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.221324 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.221403 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.221419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.222989 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.222991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.224909 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.224920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.226109 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.226113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.226198 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.226253 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.226308 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.226375 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12200,"thread_start_us":95,"threads_started":1}
I20260924 16:09:21.227512 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.208454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.13.8.67:39375)
0924 16:09:21.215110 (+ 6656us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.215130 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.220690 (+ 5560us) server_negotiation.cc:407] Connection header received
0924 16:09:21.220778 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.220784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.220990 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.221130 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.221588 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.222844 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.225075 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.225953 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.226994 (+ 1041us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.227156 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.227250 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.227379 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6535,"thread_start_us":121,"threads_started":1}
I20260924 16:09:21.230610 26617 tablet_service.cc:1467] Processing CreateTablet for tablet 6fafd9dcc09545aca7993e5429832244 (DEFAULT_TABLE table=test-table [id=32546ebdee5647ba84434280ddda8a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:21.231693 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fafd9dcc09545aca7993e5429832244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:21.236291 26719 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Bootstrap starting.
I20260924 16:09:21.238648 26718 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Bootstrap starting.
I20260924 16:09:21.240525 26720 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Bootstrap starting.
I20260924 16:09:21.240657 26719 tablet_bootstrap.cc:654] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:21.242609 26720 tablet_bootstrap.cc:654] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:21.242633 26718 tablet_bootstrap.cc:654] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:21.243440 26720 log.cc:824] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:21.243680 26719 log.cc:824] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:21.244660 26718 log.cc:824] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:21.246035 26720 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: No bootstrap required, opened a new log
I20260924 16:09:21.246253 26720 ts_tablet_manager.cc:1397] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:09:21.247241 26719 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: No bootstrap required, opened a new log
I20260924 16:09:21.247434 26719 ts_tablet_manager.cc:1397] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260924 16:09:21.250083 26720 raft_consensus.cc:348] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.250389 26720 raft_consensus.cc:374] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:21.250456 26720 raft_consensus.cc:729] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f531d27908364208af80dfeb7c490867, State: Initialized, Role: FOLLOWER
I20260924 16:09:21.250993 26720 consensus_queue.cc:260] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.252149 26720 ts_tablet_manager.cc:1428] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:09:21.252580 26684 heartbeater.cc:503] Master 127.13.8.126:44159 was elected leader, sending a full tablet report...
I20260924 16:09:21.252440 26719 raft_consensus.cc:348] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.252725 26719 raft_consensus.cc:374] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:21.252796 26719 raft_consensus.cc:729] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001301b466924d609503bfd4d61a2dc0, State: Initialized, Role: FOLLOWER
I20260924 16:09:21.253311 26719 consensus_queue.cc:260] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.254894 26719 ts_tablet_manager.cc:1428] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260924 16:09:21.257135 26718 tablet_bootstrap.cc:492] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: No bootstrap required, opened a new log
I20260924 16:09:21.257359 26718 ts_tablet_manager.cc:1397] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260924 16:09:21.261521 26718 raft_consensus.cc:348] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.261821 26718 raft_consensus.cc:374] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:21.261893 26718 raft_consensus.cc:729] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9dd0af246184f39984f3ede61402269, State: Initialized, Role: FOLLOWER
I20260924 16:09:21.262367 26549 heartbeater.cc:503] Master 127.13.8.126:44159 was elected leader, sending a full tablet report...
I20260924 16:09:21.266172 26718 consensus_queue.cc:260] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.267674 26718 ts_tablet_manager.cc:1428] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260924 16:09:21.268056 26414 heartbeater.cc:503] Master 127.13.8.126:44159 was elected leader, sending a full tablet report...
W20260924 16:09:21.371060 26550 tablet.cc:2369] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:09:21.375133 26415 tablet.cc:2369] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:09:21.378314 26685 tablet.cc:2369] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:09:21.503054 26724 raft_consensus.cc:482] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:21.503357 26724 raft_consensus.cc:504] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.504976 26724 leader_election.cc:288] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9dd0af246184f39984f3ede61402269 (127.13.8.65:38695), 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303)
I20260924 16:09:21.508289 26727 raft_consensus.cc:482] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:21.508512 26727 raft_consensus.cc:504] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.509832 26727 leader_election.cc:288] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303), f531d27908364208af80dfeb7c490867 (127.13.8.67:39375)
I20260924 16:09:21.510389 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.505316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:55393 (local address 127.13.8.65:38695)
0924 16:09:21.505462 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.505467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.505724 (+ 257us) server_negotiation.cc:407] Connection header received
0924 16:09:21.505937 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.505940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.505999 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.506085 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.506445 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.507377 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.509063 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.509674 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.509761 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.509826 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.510151 (+ 325us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.510263 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:09:21.510641 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.505670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:57153 (local address 127.13.8.66:39303)
0924 16:09:21.505797 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.505802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.505907 (+ 105us) server_negotiation.cc:407] Connection header received
0924 16:09:21.506129 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.506132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.506187 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.506268 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.506720 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.507521 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.509164 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.509761 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.509851 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.509924 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.510403 (+ 479us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.510505 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:09:21.510790 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.505213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:38695 (local address 127.13.8.67:55393)
0924 16:09:21.505628 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.505644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.505816 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.506117 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.506120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.506139 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.506320 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.506326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.507492 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.507495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.508913 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.508924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.509920 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.509923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.510018 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.510039 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.510538 (+ 499us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.510627 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":70,"thread_start_us":129,"threads_started":1}
I20260924 16:09:21.511420 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.505231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39303 (local address 127.13.8.67:57153)
0924 16:09:21.505819 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.505833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.505989 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.506440 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.506443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.506461 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.506580 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.506587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.507646 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.507648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.509006 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.509017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.510104 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.510109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.510208 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.510274 (+ 66us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.511216 (+ 942us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.511286 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260924 16:09:21.512010 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f531d27908364208af80dfeb7c490867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9dd0af246184f39984f3ede61402269" is_pre_election: true
I20260924 16:09:21.512256 26360 raft_consensus.cc:2454] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f531d27908364208af80dfeb7c490867 in term 0.
I20260924 16:09:21.512704 26501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f531d27908364208af80dfeb7c490867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001301b466924d609503bfd4d61a2dc0" is_pre_election: true
I20260924 16:09:21.512759 26570 leader_election.cc:302] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f531d27908364208af80dfeb7c490867, f9dd0af246184f39984f3ede61402269; no voters:
I20260924 16:09:21.512992 26501 raft_consensus.cc:2454] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f531d27908364208af80dfeb7c490867 in term 0.
I20260924 16:09:21.513085 26724 raft_consensus.cc:2781] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:09:21.513195 26724 raft_consensus.cc:482] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:09:21.513278 26724 raft_consensus.cc:3037] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:21.514855 26724 raft_consensus.cc:504] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.515717 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f531d27908364208af80dfeb7c490867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9dd0af246184f39984f3ede61402269"
I20260924 16:09:21.515836 26360 raft_consensus.cc:3037] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:21.517174 26360 raft_consensus.cc:2454] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f531d27908364208af80dfeb7c490867 in term 1.
I20260924 16:09:21.517329 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.511676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:48887 (local address 127.13.8.67:39375)
0924 16:09:21.512067 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.512072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.512084 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:09:21.512121 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.512123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.512164 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.512226 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.513691 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.514478 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.516076 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.516684 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.516760 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.516815 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.517143 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.517210 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260924 16:09:21.517670 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.511063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:39375 (local address 127.13.8.65:48887)
0924 16:09:21.511798 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.511812 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.511901 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.513389 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.513392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.513405 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.513545 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.513550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.514625 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.514628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.515882 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.515891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.516908 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.516911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.517011 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.517032 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.517442 (+ 410us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.517513 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":139,"threads_started":1}
I20260924 16:09:21.517760 26570 leader_election.cc:302] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f531d27908364208af80dfeb7c490867, f9dd0af246184f39984f3ede61402269; no voters:
I20260924 16:09:21.517990 26724 leader_election.cc:288] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [CANDIDATE]: Term 1 election: Requested vote from peers f9dd0af246184f39984f3ede61402269 (127.13.8.65:38695), 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303)
I20260924 16:09:21.518080 26724 raft_consensus.cc:2781] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:21.518148 26724 raft_consensus.cc:686] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 1 LEADER]: Becoming Leader. State: Replica: f531d27908364208af80dfeb7c490867, State: Running, Role: LEADER
I20260924 16:09:21.518184 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.510591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:42335 (local address 127.13.8.66:39303)
0924 16:09:21.510796 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.510801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.511461 (+ 660us) server_negotiation.cc:407] Connection header received
0924 16:09:21.513407 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.513411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.513464 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.513553 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:21.514324 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.515067 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.516718 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.517394 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.517482 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.517545 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.517938 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.518039 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":144,"threads_started":1}
I20260924 16:09:21.518512 26724 consensus_queue.cc:237] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } }
I20260924 16:09:21.518827 26636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f9dd0af246184f39984f3ede61402269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f531d27908364208af80dfeb7c490867" is_pre_election: true
I20260924 16:09:21.519377 26501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f531d27908364208af80dfeb7c490867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001301b466924d609503bfd4d61a2dc0"
I20260924 16:09:21.519542 26501 raft_consensus.cc:3037] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:21.520946 26501 raft_consensus.cc:2454] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f531d27908364208af80dfeb7c490867 in term 1.
I20260924 16:09:21.522279 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.510177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39303 (local address 127.13.8.65:42335)
0924 16:09:21.511369 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.511385 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:21.513281 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:21.513960 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:21.513963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:21.513998 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:21.514151 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.514157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.515222 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.515227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.516564 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:21.516574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.517650 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.517653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:21.517755 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.517787 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 16:09:21.522010 (+ 4223us) client_negotiation.cc:240] Negotiation successful
0924 16:09:21.522104 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"thread_start_us":139,"threads_started":1}
I20260924 16:09:21.522699 26501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fafd9dcc09545aca7993e5429832244" candidate_uuid: "f9dd0af246184f39984f3ede61402269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001301b466924d609503bfd4d61a2dc0" is_pre_election: true
I20260924 16:09:21.522897 26501 raft_consensus.cc:2379] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9dd0af246184f39984f3ede61402269 in current term 1: Already voted for candidate f531d27908364208af80dfeb7c490867 in this term.
I20260924 16:09:21.522491 26213 catalog_manager.cc:5495] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 reported cstate change: term changed from 0 to 1, leader changed from <none> to f531d27908364208af80dfeb7c490867 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "f531d27908364208af80dfeb7c490867" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f531d27908364208af80dfeb7c490867" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 39375 } health_report { overall_health: HEALTHY } } }
I20260924 16:09:21.523387 26300 leader_election.cc:302] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9dd0af246184f39984f3ede61402269; no voters: 001301b466924d609503bfd4d61a2dc0, f531d27908364208af80dfeb7c490867
I20260924 16:09:21.523873 26727 raft_consensus.cc:2726] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 16:09:21.573635 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.569816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.13.8.67:39375)
0924 16:09:21.569963 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:21.569967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:21.569984 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:09:21.570092 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:21.570095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:21.570160 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:21.570322 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:09:21.570588 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.571104 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.572028 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:21.572274 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:21.572390 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:21.572736 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:09:21.573215 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:09:21.573363 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:21.573400 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 16:09:21.573459 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 16:09:21.579945 26360 raft_consensus.cc:1261] T 6fafd9dcc09545aca7993e5429832244 P f9dd0af246184f39984f3ede61402269 [term 1 FOLLOWER]: Refusing update from remote peer f531d27908364208af80dfeb7c490867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:21.579947 26501 raft_consensus.cc:1261] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 1 FOLLOWER]: Refusing update from remote peer f531d27908364208af80dfeb7c490867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:21.580912 26740 consensus_queue.cc:1035] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9dd0af246184f39984f3ede61402269" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 38695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:21.581128 26724 consensus_queue.cc:1035] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001301b466924d609503bfd4d61a2dc0" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:21.589315 26743 mvcc.cc:205] Tried to move back new op lower bound from 7332930197823361024 to 7332930197582598144. Current Snapshot: MvccSnapshot[applied={T|T < 7332930197823361024}]
I20260924 16:09:21.590785 26744 mvcc.cc:205] Tried to move back new op lower bound from 7332930197823361024 to 7332930197582598144. Current Snapshot: MvccSnapshot[applied={T|T < 7332930197823361024}]
I20260924 16:09:21.597280 26745 mvcc.cc:205] Tried to move back new op lower bound from 7332930197823361024 to 7332930197582598144. Current Snapshot: MvccSnapshot[applied={T|T < 7332930197823361024 or (T in {7332930197823361024})}]
I20260924 16:09:21.602661 26212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:21.602849 26212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:21.605638 26212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 204c18a1d9ef458bbbd86a6e51f7186d: Sending DeleteTablet for 3 replicas of tablet 6fafd9dcc09545aca7993e5429832244
I20260924 16:09:21.606526 26340 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fafd9dcc09545aca7993e5429832244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:21 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:42736
I20260924 16:09:21.606627 26617 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fafd9dcc09545aca7993e5429832244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:21 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51350
I20260924 16:09:21.607311 26754 tablet_replica.cc:330] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: stopping tablet replica
I20260924 16:09:21.607393 26482 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fafd9dcc09545aca7993e5429832244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:21 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:57318
I20260924 16:09:21.607645 26754 raft_consensus.cc:2229] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:09:21.608071 26754 raft_consensus.cc:2258] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:21.608675 26755 tablet_replica.cc:330] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: stopping tablet replica
I20260924 16:09:21.608966 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26251
I20260924 16:09:21.609264 26755 raft_consensus.cc:2229] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:21.609779 26755 raft_consensus.cc:2258] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:21.611756 26755 ts_tablet_manager.cc:1905] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:21.614614 26755 ts_tablet_manager.cc:1918] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:21.614768 26755 log.cc:1196] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/6fafd9dcc09545aca7993e5429832244
I20260924 16:09:21.615170 26755 ts_tablet_manager.cc:1939] T 6fafd9dcc09545aca7993e5429832244 P 001301b466924d609503bfd4d61a2dc0: Deleting consensus metadata
I20260924 16:09:21.616587 26198 catalog_manager.cc:4841] TS 001301b466924d609503bfd4d61a2dc0 (127.13.8.66:39303): tablet 6fafd9dcc09545aca7993e5429832244 (table test-table [id=32546ebdee5647ba84434280ddda8a4d]) successfully deleted
I20260924 16:09:21.617411 26754 ts_tablet_manager.cc:1905] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:21.620143 26754 ts_tablet_manager.cc:1918] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:21.620242 26754 log.cc:1196] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/6fafd9dcc09545aca7993e5429832244
I20260924 16:09:21.620563 26754 ts_tablet_manager.cc:1939] T 6fafd9dcc09545aca7993e5429832244 P f531d27908364208af80dfeb7c490867: Deleting consensus metadata
I20260924 16:09:21.621843 26200 catalog_manager.cc:4841] TS f531d27908364208af80dfeb7c490867 (127.13.8.67:39375): tablet 6fafd9dcc09545aca7993e5429832244 (table test-table [id=32546ebdee5647ba84434280ddda8a4d]) successfully deleted
W20260924 16:09:21.627035 26197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:38695 (error 108)
I20260924 16:09:21.627916 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26416
I20260924 16:09:21.628161 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:21.627845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:38695 (local address 127.0.0.1:42752)
0924 16:09:21.627982 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:21.628061 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 16:09:21.628834 26197 catalog_manager.cc:4584] TS f9dd0af246184f39984f3ede61402269 (127.13.8.65:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fafd9dcc09545aca7993e5429832244: Network error: Client connection negotiation failed: client connection to 127.13.8.65:38695: connect: Connection refused (error 111)
I20260924 16:09:21.636595 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26552
I20260924 16:09:21.643173 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26180
I20260924 16:09:21.651469 13345 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 16:09:21.996546 13345 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 16:09:21.996666 13345 mini_ranger.cc:66] Stopping Ranger...
I20260924 16:09:22.342696 13345 mini_ranger.cc:68] Stopped Ranger
2026-09-24 16:09:22.342 UTC [25232] LOG: received smart shutdown request
2026-09-24 16:09:22.347 UTC [25232] LOG: background worker "logical replication launcher" (PID 25241) exited with exit code 1
2026-09-24 16:09:22.347 UTC [25236] LOG: shutting down
2026-09-24 16:09:22.533 UTC [25232] LOG: database system is shut down
2026-09-24T16:09:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49366 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:09:22 dist-test-slave-vj6n krb5kdc[26765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:09:22 dist-test-slave-vj6n krb5kdc[26765](info): set up 2 sockets
Sep 24 16:09:22 dist-test-slave-vj6n krb5kdc[26765](info): commencing operation
krb5kdc: starting...
W20260924 16:09:23.711004 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:09:23 dist-test-slave-vj6n krb5kdc[26765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:09:23.833722 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:38355
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:23.946305 26778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:23.946583 26778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:23.946645 26778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:23.950819 26778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:23.950879 26778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:23.950904 26778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:23.950924 26778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:23.950942 26778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:23.956249 26778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:38355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:23.957237 26778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:23.958343 26778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:23.967082 26786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:23.967056 26783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:23.967056 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:23.967214 26778 server_base.cc:1034] running on GCE node
I20260924 16:09:23.968011 26778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:23.969089 26778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:23.970309 26778 hybrid_clock.cc:648] HybridClock initialized: now 1790266163970298 us; error 31 us; skew 500 ppm
Sep 24 16:09:23 dist-test-slave-vj6n krb5kdc[26765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:23.973006 26778 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:23.973803 26778 webserver.cc:460] Webserver started at http://127.13.8.126:43813/ using document root <none> and password file <none>
I20260924 16:09:23.974382 26778 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:23.974447 26778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:23.974613 26778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:23.976569 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "f9a82c37ab1d4f489802d207d40a4703"
format_stamp: "Formatted at 2026-09-24 16:09:23 on dist-test-slave-vj6n"
server_key: "7ef74f2707445b281dd7cb77fd686a44"
server_key_iv: "87f08c1acad373e19786ae38ec71c49b"
server_key_version: "encryptionkey@0"
I20260924 16:09:23.977347 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "f9a82c37ab1d4f489802d207d40a4703"
format_stamp: "Formatted at 2026-09-24 16:09:23 on dist-test-slave-vj6n"
server_key: "7ef74f2707445b281dd7cb77fd686a44"
server_key_iv: "87f08c1acad373e19786ae38ec71c49b"
server_key_version: "encryptionkey@0"
I20260924 16:09:23.981379 26778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:09:23.983850 26793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:23.985052 26778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:23.985194 26778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "f9a82c37ab1d4f489802d207d40a4703"
format_stamp: "Formatted at 2026-09-24 16:09:23 on dist-test-slave-vj6n"
server_key: "7ef74f2707445b281dd7cb77fd686a44"
server_key_iv: "87f08c1acad373e19786ae38ec71c49b"
server_key_version: "encryptionkey@0"
I20260924 16:09:23.985307 26778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:24.008289 26778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:24.011677 26778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:24.011902 26778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:24.019536 26778 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:38355
I20260924 16:09:24.019553 26846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:38355 every 8 connection(s)
I20260924 16:09:24.020527 26778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:24.023453 26847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:24.027557 26847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.029861 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26778
I20260924 16:09:24.030037 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:09:24.030411 13345 external_mini_cluster.cc:1428] Setting key 54dd650d2d6e710237fde15dd742406e
WARNING: no policy specified for kudu/127.13.8.125@KRBTEST.COM; defaulting to no policy
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.125@KRBTEST.COM" created.
I20260924 16:09:24.045259 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.029959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38355 (local address 127.0.0.1:52134)
0924 16:09:24.030655 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.030718 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.032025 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.032954 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.032959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.033298 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.033579 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.033593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.035406 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.035410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.036233 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.036243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.037210 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.037214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.037329 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.037965 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.037981 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.039667 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.041690 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.041700 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.041715 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.041937 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.042600 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.042602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.042604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.042715 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.043512 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.043517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.043644 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.043832 (+ 188us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.044199 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":107,"spinlock_wait_cycles":6400,"thread_start_us":190,"threads_started":1}
I20260924 16:09:24.045259 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.030921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.13.8.126:38355)
0924 16:09:24.031210 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.031220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.031255 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:09:24.032437 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.032446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.032716 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.032926 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.033744 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.035245 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.036377 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.037037 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.037214 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.039813 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.039819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.039831 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.039854 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.041558 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.042305 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.042312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.042315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.042481 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.042813 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.042820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.042825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.043368 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.043559 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.043896 (+ 337us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.044148 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":183,"spinlock_wait_cycles":3456,"thread_start_us":119,"threads_started":1}
I20260924 16:09:24.047981 26847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.048792 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.048520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60438)
0924 16:09:24.048644 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.048724 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 16:09:24.049093 26795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
I20260924 16:09:24.050033 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.049816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60444)
0924 16:09:24.049927 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.049977 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 16:09:24.050298 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43339: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
I20260924 16:09:24.059149 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:43339
--webserver_interface=127.13.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:24.099552 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } attempt: 1
I20260924 16:09:24.100526 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.100187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60458)
0924 16:09:24.100350 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.100428 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 16:09:24.101294 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.101050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60464)
0924 16:09:24.101184 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.101239 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 16:09:24.101418 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43339: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
I20260924 16:09:24.169655 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } attempt: 2
I20260924 16:09:24.170532 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.170211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60470)
0924 16:09:24.170393 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.170457 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 16:09:24.171284 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.171058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60476)
0924 16:09:24.171181 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.171231 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 16:09:24.171399 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43339: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43339: connect: Connection refused (error 111)
W20260924 16:09:24.172367 26854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:24.172646 26854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:24.172751 26854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:24.176761 26854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:24.176844 26854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:24.176892 26854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:24.176935 26854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:24.176970 26854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:24.182332 26854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:43339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:24.183404 26854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:24.184486 26854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:24.193329 26860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:24.193385 26862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:24.193385 26859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:24.193552 26854 server_base.cc:1034] running on GCE node
I20260924 16:09:24.194311 26854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:24.195384 26854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:24.196550 26854 hybrid_clock.cc:648] HybridClock initialized: now 1790266164196518 us; error 42 us; skew 500 ppm
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:24.199250 26854 init.cc:377] Logged in from keytab as kudu/127.13.8.125@KRBTEST.COM (short username kudu)
I20260924 16:09:24.200124 26854 webserver.cc:460] Webserver started at http://127.13.8.125:38567/ using document root <none> and password file <none>
I20260924 16:09:24.200685 26854 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:24.200745 26854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:24.200991 26854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:24.203052 26854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data/instance:
uuid: "77006fcd0ebb4537858ac985d2fee5e4"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "85251bb0d0607f9212182c5f3b923be4"
server_key_iv: "8f0d6c376e8bbd7fd30ae1a777ba6813"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.203574 26854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal/instance:
uuid: "77006fcd0ebb4537858ac985d2fee5e4"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "85251bb0d0607f9212182c5f3b923be4"
server_key_iv: "8f0d6c376e8bbd7fd30ae1a777ba6813"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.207185 26854 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:09:24.209532 26869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:24.210803 26854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:09:24.210954 26854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "77006fcd0ebb4537858ac985d2fee5e4"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "85251bb0d0607f9212182c5f3b923be4"
server_key_iv: "8f0d6c376e8bbd7fd30ae1a777ba6813"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.211071 26854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:24.233012 26854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:24.235775 26854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:24.235975 26854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:24.244335 26854 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.125:43339
I20260924 16:09:24.244450 26922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.125:43339 every 8 connection(s)
I20260924 16:09:24.245579 26854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
I20260924 16:09:24.248744 26923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:24.252483 26923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.255613 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26854
I20260924 16:09:24.255765 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-1/wal/instance
I20260924 16:09:24.256045 13345 external_mini_cluster.cc:1428] Setting key af0f319afa4a55b83832067511b811ce
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:24.262557 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } attempt: 3
WARNING: no policy specified for kudu/127.13.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.124@KRBTEST.COM" created.
I20260924 16:09:24.267030 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.254300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.13.8.126:38355)
0924 16:09:24.254461 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.254465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.255240 (+ 775us) server_negotiation.cc:407] Connection header received
0924 16:09:24.256328 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.256332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.256390 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.256498 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.257831 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.258430 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.259590 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.260032 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.260150 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.262710 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.262713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.262716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.262741 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.264549 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.265149 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.265153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.265155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.265247 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.265564 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.265567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.265569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.265802 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.265875 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.266626 (+ 751us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.266756 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:09:24.267956 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.254660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38355 (local address 127.0.0.1:52138)
0924 16:09:24.255084 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.255135 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.256061 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.256675 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.256682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.257002 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.257664 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.257685 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.258576 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.258608 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.259459 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.259468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.260185 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.260188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.260320 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.260895 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.260912 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.262531 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.264697 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.264701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.264713 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.265052 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.265343 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.265345 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.265346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.265471 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.265941 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.265947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.266318 (+ 371us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.266559 (+ 241us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.266872 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":101,"threads_started":1}
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:09:24.270673 26923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.273658 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.263019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60492)
0924 16:09:24.263211 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.263226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.263305 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.264210 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.264212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.264403 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.264515 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.264520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.265767 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.265770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.266583 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.266593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.267302 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.267306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.267404 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.267898 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.267911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.269253 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.271970 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.271976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.271981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.272265 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.272594 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.272597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.272599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.272686 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.273141 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.273145 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.273249 (+ 104us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.273366 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.273489 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260924 16:09:24.273761 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.263145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.13.8.125:43339)
0924 16:09:24.263712 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.263721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.263769 (+ 48us) server_negotiation.cc:407] Connection header received
0924 16:09:24.263815 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.263819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.264032 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.264199 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.264635 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.265648 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.266710 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.267164 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.267627 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.269393 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.269401 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.269411 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.269431 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.271845 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.272389 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.272394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.272396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.272488 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.272781 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.272785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.272787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.273028 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.273186 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.273450 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.273582 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":92,"threads_started":1}
I20260924 16:09:24.275799 26847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.276574 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.276313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44564)
0924 16:09:24.276434 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.276481 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260924 16:09:24.277333 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.277063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44570)
0924 16:09:24.277226 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.277276 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260924 16:09:24.277515 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:09:24.282207 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.271185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60498)
0924 16:09:24.271378 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.271395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.271474 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.272111 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.272114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.272271 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.272375 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.272382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.273574 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.273577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.274459 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.274472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.275385 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.275388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.275521 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.275983 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.276006 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.278779 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.280808 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.280811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.280812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.280994 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.281355 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.281358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.281359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.281432 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.281844 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.281846 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.281925 (+ 79us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.282013 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.282103 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":20}
I20260924 16:09:24.282457 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.271268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.13.8.125:43339)
0924 16:09:24.271886 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.271899 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.271914 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:09:24.271951 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.271953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.272004 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.272101 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.272510 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.273453 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.274593 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.275260 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.275376 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.278919 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.278923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.278925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.278945 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.280689 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.281116 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.281122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.281124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.281261 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.281520 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.281522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.281523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.281739 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.281817 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.282073 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.282164 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":254,"thread_start_us":121,"threads_started":1}
I20260924 16:09:24.283145 26923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.283762 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.283544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44572)
0924 16:09:24.283649 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.283714 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 16:09:24.283967 26872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
I20260924 16:09:24.284771 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.284600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44580)
0924 16:09:24.284689 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.284729 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 16:09:24.284926 26923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
I20260924 16:09:24.285974 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:37791
--webserver_interface=127.13.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:24.308796 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 1
I20260924 16:09:24.309629 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.309327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44582)
0924 16:09:24.309485 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.309544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 16:09:24.310397 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.310138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44586)
0924 16:09:24.310290 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.310342 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 16:09:24.310523 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
I20260924 16:09:24.334196 26923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 1
I20260924 16:09:24.335155 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.334847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44598)
0924 16:09:24.335004 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.335065 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260924 16:09:24.335877 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.335669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44604)
0924 16:09:24.335776 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.335827 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 16:09:24.336004 26923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
I20260924 16:09:24.385697 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 2
I20260924 16:09:24.386545 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.386187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44612)
0924 16:09:24.386392 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.386461 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260924 16:09:24.387300 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.387041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44620)
0924 16:09:24.387193 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.387250 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260924 16:09:24.387432 26847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
W20260924 16:09:24.398021 26931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:24.398367 26931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:24.398478 26931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:24.402518 26931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:24.402597 26931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:24.402634 26931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:24.402667 26931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:24.402697 26931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:24.404153 26923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 2
I20260924 16:09:24.404954 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.404666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44624)
0924 16:09:24.404816 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.404883 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260924 16:09:24.405652 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.405439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44630)
0924 16:09:24.405546 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.405593 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 16:09:24.405786 26923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:37791: Network error: Client connection negotiation failed: client connection to 127.13.8.124:37791: connect: Connection refused (error 111)
I20260924 16:09:24.408254 26931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:38355,127.13.8.125:43339,127.13.8.124:37791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:37791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:24.409293 26931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:24.410329 26931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:24.418655 26936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:24.418738 26937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:24.418781 26931 server_base.cc:1034] running on GCE node
W20260924 16:09:24.418668 26939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:24.419515 26931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:24.420641 26931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:24.421837 26931 hybrid_clock.cc:648] HybridClock initialized: now 1790266164421803 us; error 45 us; skew 500 ppm
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:24.424726 26931 init.cc:377] Logged in from keytab as kudu/127.13.8.124@KRBTEST.COM (short username kudu)
I20260924 16:09:24.425721 26931 webserver.cc:460] Webserver started at http://127.13.8.124:45347/ using document root <none> and password file <none>
I20260924 16:09:24.426340 26931 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:24.426409 26931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:24.426599 26931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:24.428623 26931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data/instance:
uuid: "28dad19a05a64e679ad02673576b4874"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "2595b050c2753bb6902cbdf675cfc810"
server_key_iv: "68330c6a0f02125e9e5788113a676d83"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.429107 26931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal/instance:
uuid: "28dad19a05a64e679ad02673576b4874"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "2595b050c2753bb6902cbdf675cfc810"
server_key_iv: "68330c6a0f02125e9e5788113a676d83"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.432789 26931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260924 16:09:24.435166 26946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:24.436394 26931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:09:24.436548 26931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "28dad19a05a64e679ad02673576b4874"
format_stamp: "Formatted at 2026-09-24 16:09:24 on dist-test-slave-vj6n"
server_key: "2595b050c2753bb6902cbdf675cfc810"
server_key_iv: "68330c6a0f02125e9e5788113a676d83"
server_key_version: "encryptionkey@0"
I20260924 16:09:24.436655 26931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:24.452248 26931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:24.455255 26931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:24.455461 26931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:24.462744 26931 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.124:37791
I20260924 16:09:24.462761 26998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.124:37791 every 8 connection(s)
I20260924 16:09:24.463683 26931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
I20260924 16:09:24.466440 26999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:24.470016 26999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.472108 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 26931
I20260924 16:09:24.472236 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790266001279336-13345-0/minicluster-data/master-2/wal/instance
I20260924 16:09:24.472498 13345 external_mini_cluster.cc:1428] Setting key 0fbf9a7ae85f119cba0697dc5fe5e23a
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:09:24.484347 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.472274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.13.8.126:38355)
0924 16:09:24.472424 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.472429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.473399 (+ 970us) server_negotiation.cc:407] Connection header received
0924 16:09:24.474514 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.474517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.474567 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.474674 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.475742 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.476526 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.477866 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.478184 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.478312 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.480852 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.480856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.480858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.480885 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.482212 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.482784 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.482789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.482791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.482876 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.483223 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.483228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.483230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.483453 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.483577 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.484068 (+ 491us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.484179 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:09:24.485070 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.473690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44632 (local address 127.13.8.124:37791)
0924 16:09:24.473978 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.473987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.474019 (+ 32us) server_negotiation.cc:407] Connection header received
0924 16:09:24.474311 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.474338 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.474565 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.474743 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.475591 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.476819 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.477952 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.478554 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.478633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.481367 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.481372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.481382 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.481402 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.483089 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.483539 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.483549 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.483550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.483624 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.483917 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.483921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.483923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.484143 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.484330 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.484833 (+ 503us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.484951 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260924 16:09:24.485072 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.472743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38355 (local address 127.0.0.1:52146)
0924 16:09:24.473244 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.473294 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.474315 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.474758 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.474763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.475241 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.475609 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.475622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.476637 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.476639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.477749 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.477760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.478354 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.478357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.478506 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.479088 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.479114 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.480674 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.482372 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.482382 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.482391 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.482678 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.482984 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.482987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.482989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.483122 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.483577 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.483584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.483825 (+ 241us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.484015 (+ 190us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.484301 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":147,"threads_started":1}
I20260924 16:09:24.486582 26847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 3
I20260924 16:09:24.487007 26999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:24.496961 26923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } attempt: 3
I20260924 16:09:24.497022 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.486981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44634)
0924 16:09:24.487379 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.487395 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.487467 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.488011 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.488014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.488194 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.488306 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.488316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.489404 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.489407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.490197 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.490209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.490800 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.490882 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.491411 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.491422 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.493022 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.495173 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.495178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.495182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.495593 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.495904 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.495907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.495909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.496003 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.496477 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.496482 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.496587 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.496694 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.496831 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320}
I20260924 16:09:24.497035 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.487195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.13.8.124:37791)
0924 16:09:24.487804 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.487808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.487819 (+ 11us) server_negotiation.cc:407] Connection header received
0924 16:09:24.487853 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.487857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.487896 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.487971 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.488443 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.489295 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490317 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.490675 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.491060 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.493156 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.493161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.493164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.493188 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.495041 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.495696 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.495700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.495702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.495785 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.496100 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.496104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.496105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.496353 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.496492 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.496759 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.496868 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535}
I20260924 16:09:24.497996 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.487712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.13.8.125:43339)
0924 16:09:24.487842 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.487846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.487858 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:09:24.487899 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.487902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.487953 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.488024 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.488514 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.489156 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490276 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.490576 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490696 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.493587 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.493592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.493599 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.493622 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.495357 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.495841 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.495845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.495846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.495929 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.496251 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.496254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.496256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.496477 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.496568 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.496843 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.497857 (+ 1014us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:09:24.498164 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.487420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52150 (local address 127.13.8.126:38355)
0924 16:09:24.487537 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.487542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.487590 (+ 48us) server_negotiation.cc:407] Connection header received
0924 16:09:24.487756 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.487760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.487803 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.487875 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.488726 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.489457 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490377 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.490677 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490780 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.494042 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.494045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.494048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.494074 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.495731 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.496428 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.496431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.496433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.496515 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.496830 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.496834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.496835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.497038 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.497253 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.497564 (+ 311us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.497668 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:09:24.498567 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.487430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43339 (local address 127.0.0.1:60504)
0924 16:09:24.487712 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.487727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.487792 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.488137 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.488140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.488262 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.488342 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.488347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.489254 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.489259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.490126 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.490137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.490693 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.490697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.490797 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.491259 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.491277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.493417 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.495475 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.495481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.495483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.495736 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.496024 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.496029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.496030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.496124 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.496552 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.496555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.496636 (+ 81us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.498295 (+ 1659us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.498422 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1}
I20260924 16:09:24.500981 26847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: Bootstrap starting.
I20260924 16:09:24.503140 26999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:09:24.504133 26847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: Neither blocks nor log segments found. Creating new log.
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:09:24.505151 26847 log.cc:824] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:24.507532 26847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: No bootstrap required, opened a new log
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
Sep 24 16:09:24 dist-test-slave-vj6n krb5kdc[26765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266164, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:09:24.512363 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.499483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.13.8.125:43339)
0924 16:09:24.499573 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.499577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.499605 (+ 28us) server_negotiation.cc:407] Connection header received
0924 16:09:24.499697 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.499699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.499730 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.499798 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.500497 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.500953 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.501923 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.502125 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.502196 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.506691 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.506695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.506698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.506724 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.509076 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.509546 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.509549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.509551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.509646 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.510267 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.510273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.510277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.510777 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.511241 (+ 464us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.511897 (+ 656us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.512010 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 16:09:24.512341 26847 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.512589 26847 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:24.512670 26847 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a82c37ab1d4f489802d207d40a4703, State: Initialized, Role: FOLLOWER
I20260924 16:09:24.513237 26847 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.514587 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.503627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44652)
0924 16:09:24.503783 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.503798 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.503869 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.504196 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.504198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.504372 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.504453 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.504459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.506148 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.506154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.507441 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.507455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.508339 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.508346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.508436 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.508937 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.508955 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.510419 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.512980 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.512982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.512984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.513226 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.513566 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.513569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.513571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.513662 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.514053 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.514056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.514155 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.514280 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.514403 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":1}
I20260924 16:09:24.515138 26847 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:24.515317 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.503651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.13.8.124:37791)
0924 16:09:24.503986 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.504011 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.504030 (+ 19us) server_negotiation.cc:407] Connection header received
0924 16:09:24.504054 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.504055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.504083 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.504185 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.504939 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.506032 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.507568 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.508205 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.508312 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.510553 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.510559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.510565 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.510591 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.512874 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.513332 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.513335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.513336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.513462 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.513752 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.513754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.513756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.513954 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.514874 (+ 920us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.514949 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.515074 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
I20260924 16:09:24.515720 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.497645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:37791 (local address 127.0.0.1:44644)
0924 16:09:24.497777 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:24.497792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:24.497863 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:24.502799 (+ 4936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:24.502802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:24.502967 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:24.503074 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.503081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.505164 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.505171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.505829 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:24.505838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.506527 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.506531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:24.506611 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.507067 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:24.507082 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:24.509492 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:24.511609 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.511615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.511620 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.511898 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.513768 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:24.513771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:24.513773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:24.514365 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:24.514823 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:24.514826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:24.514913 (+ 87us) client_negotiation.cc:769] Sending connection context
0924 16:09:24.515022 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 16:09:24.515307 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"spinlock_wait_cycles":1280}
I20260924 16:09:24.516573 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.497995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44644 (local address 127.13.8.124:37791)
0924 16:09:24.502555 (+ 4560us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.502560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.502572 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:09:24.502606 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.502610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.502654 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.502770 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.504288 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.505031 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.505995 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.506380 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.506813 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.509637 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.509641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.509644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.509667 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.511452 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.513321 (+ 1869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.513324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.513326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.513428 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.514504 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.514507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.514508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.514714 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.515969 (+ 1255us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.516044 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.516175 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4473,"server-negotiator.queue_time_us":4491}
I20260924 16:09:24.518036 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:09:24.518636 26999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: Bootstrap starting.
I20260924 16:09:24.519711 26923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: Bootstrap starting.
I20260924 16:09:24.521335 26999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:24.522053 26999 log.cc:824] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:24.522166 26923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:24.523051 26923 log.cc:824] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:24.524200 26999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: No bootstrap required, opened a new log
I20260924 16:09:24.520655 26778 master_runner.cc:186] Error getting master registration for 127.13.8.125:43339: OK, instance_id { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" instance_seqno: 1790266164241664 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 16:09:24.524760 26778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:09:24.520563 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.525002 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.525032 26923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: No bootstrap required, opened a new log
W20260924 16:09:24.527341 27017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:09:24.527455 27017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:09:24.528263 26999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.528566 26999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:24.528666 26999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28dad19a05a64e679ad02673576b4874, State: Initialized, Role: FOLLOWER
I20260924 16:09:24.528916 26923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.529165 26923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:24.529289 26923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77006fcd0ebb4537858ac985d2fee5e4, State: Initialized, Role: FOLLOWER
I20260924 16:09:24.529269 26999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.529793 26923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.530652 26999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:24.531078 26923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 16:09:24.536983 27046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:09:24.537117 27046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:09:24.537796 27029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.537973 27029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.538476 27028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.538712 27028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.543195 26931 master_runner.cc:263] No leader master found from master 28dad19a05a64e679ad02673576b4874
I20260924 16:09:24.543253 26931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:09:24.543901 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.525898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.13.8.124:37791)
0924 16:09:24.526007 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.526010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.527802 (+ 1792us) server_negotiation.cc:407] Connection header received
0924 16:09:24.527986 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.527991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.528039 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.528091 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.529108 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.529884 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.533848 (+ 3964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.534198 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.534293 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.539635 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.539639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.539642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.539670 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.541892 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.542567 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.542571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.542573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.542656 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.542954 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.542957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.542959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.543164 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.543284 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.543633 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.543749 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 16:09:24.545882 26854 master_runner.cc:263] No leader master found from master 77006fcd0ebb4537858ac985d2fee5e4
I20260924 16:09:24.545969 26854 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:09:24.546365 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.526884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.13.8.125:43339)
0924 16:09:24.526992 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.526997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.531451 (+ 4454us) server_negotiation.cc:407] Connection header received
0924 16:09:24.532768 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.532771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.532820 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.532916 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.537753 (+ 4837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.538499 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.539578 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.539878 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.539992 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.541765 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.541768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.541774 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.541802 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.543609 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.544685 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.544689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.544690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.544773 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.545078 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.545081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.545082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.545306 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.545399 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.545733 (+ 334us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.545837 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 16:09:24.549156 27051 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:09:24.549340 27051 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:09:24.551191 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:24.525495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.13.8.126:38355)
0924 16:09:24.525596 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:24.525601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:24.531255 (+ 5654us) server_negotiation.cc:407] Connection header received
0924 16:09:24.531644 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:24.531648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:24.531692 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:24.531743 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:24.534801 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.535556 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.540065 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:24.540435 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:24.540525 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:24.547774 (+ 7249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:24.547777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:24.547780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:24.547805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:24.549372 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.549839 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.549843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.549844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.549922 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:24.550200 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:24.550203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:24.550205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:24.550441 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:24.550563 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:24.550922 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 16:09:24.551037 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 16:09:24.706983 26952 tablet.cc:2369] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:09:24.730957 27028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:24.731247 27028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.732522 27028 leader_election.cc:288] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9a82c37ab1d4f489802d207d40a4703 (127.13.8.126:38355), 77006fcd0ebb4537858ac985d2fee5e4 (127.13.8.125:43339)
I20260924 16:09:24.733230 26820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28dad19a05a64e679ad02673576b4874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a82c37ab1d4f489802d207d40a4703" is_pre_election: true
I20260924 16:09:24.733366 26894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28dad19a05a64e679ad02673576b4874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77006fcd0ebb4537858ac985d2fee5e4" is_pre_election: true
I20260924 16:09:24.733630 26894 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28dad19a05a64e679ad02673576b4874 in term 0.
I20260924 16:09:24.733630 26820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28dad19a05a64e679ad02673576b4874 in term 0.
I20260924 16:09:24.734088 26948 leader_election.cc:302] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28dad19a05a64e679ad02673576b4874, 77006fcd0ebb4537858ac985d2fee5e4; no voters:
I20260924 16:09:24.734436 27028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:09:24.734534 27028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:09:24.734578 27028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:24.735632 27028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.735955 27028 leader_election.cc:288] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [CANDIDATE]: Term 1 election: Requested vote from peers f9a82c37ab1d4f489802d207d40a4703 (127.13.8.126:38355), 77006fcd0ebb4537858ac985d2fee5e4 (127.13.8.125:43339)
I20260924 16:09:24.736420 26894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28dad19a05a64e679ad02673576b4874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77006fcd0ebb4537858ac985d2fee5e4"
I20260924 16:09:24.736554 26820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28dad19a05a64e679ad02673576b4874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a82c37ab1d4f489802d207d40a4703"
I20260924 16:09:24.736563 26894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:24.736671 26820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:24.737898 26894 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28dad19a05a64e679ad02673576b4874 in term 1.
I20260924 16:09:24.737898 26820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28dad19a05a64e679ad02673576b4874 in term 1.
W20260924 16:09:24.738178 26875 tablet.cc:2369] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:09:24.738265 26948 leader_election.cc:302] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28dad19a05a64e679ad02673576b4874, f9a82c37ab1d4f489802d207d40a4703; no voters:
I20260924 16:09:24.738796 27028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:24.739097 27028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [term 1 LEADER]: Becoming Leader. State: Replica: 28dad19a05a64e679ad02673576b4874, State: Running, Role: LEADER
I20260924 16:09:24.739485 27028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } }
I20260924 16:09:24.741734 27055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28dad19a05a64e679ad02673576b4874. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.741937 27055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:24.743646 27057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:24.744510 27057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:24.749130 26894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [term 1 FOLLOWER]: Refusing update from remote peer 28dad19a05a64e679ad02673576b4874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:24.749169 26820 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [term 1 FOLLOWER]: Refusing update from remote peer 28dad19a05a64e679ad02673576b4874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:24.749886 27055 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:24.750036 27028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:24.754448 27029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28dad19a05a64e679ad02673576b4874. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.754583 27029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.754746 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28dad19a05a64e679ad02673576b4874. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.754864 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.758814 27028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.758939 27028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:24.759795 27059 mvcc.cc:205] Tried to move back new op lower bound from 7332930210805891072 to 7332930210776203264. Current Snapshot: MvccSnapshot[applied={T|T < 7332930210805891072}]
I20260924 16:09:24.759939 27029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.760058 27029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.760365 27058 mvcc.cc:205] Tried to move back new op lower bound from 7332930210805891072 to 7332930210776203264. Current Snapshot: MvccSnapshot[applied={T|T < 7332930210805891072}]
I20260924 16:09:24.761468 27028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.761567 27028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:24.762701 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28dad19a05a64e679ad02673576b4874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a82c37ab1d4f489802d207d40a4703" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38355 } } peers { permanent_uuid: "77006fcd0ebb4537858ac985d2fee5e4" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43339 } } peers { permanent_uuid: "28dad19a05a64e679ad02673576b4874" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 37791 } } }
I20260924 16:09:24.762812 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:09:24.763612 27057 catalog_manager.cc:1347] Generated new cluster ID: aa74fa7a7a4444a6bc643be8d7de8497
I20260924 16:09:24.763676 27057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:24.783025 27057 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:09:24.784471 27057 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:24.791306 27057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874: Generated new TSK 0
I20260924 16:09:24.792124 27057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:09:25.529035 27017 catalog_manager.cc:1259] Loaded cluster ID: aa74fa7a7a4444a6bc643be8d7de8497
I20260924 16:09:25.529116 27017 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: loading cluster ID for follower catalog manager: success
I20260924 16:09:25.531068 27017 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: acquiring CA information for follower catalog manager: success
I20260924 16:09:25.532500 27017 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9a82c37ab1d4f489802d207d40a4703: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:09:25.550747 27051 catalog_manager.cc:1259] Loaded cluster ID: aa74fa7a7a4444a6bc643be8d7de8497
I20260924 16:09:25.550846 27051 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: loading cluster ID for follower catalog manager: success
I20260924 16:09:25.552496 27051 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: acquiring CA information for follower catalog manager: success
I20260924 16:09:25.553707 27051 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77006fcd0ebb4537858ac985d2fee5e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:09:25.862614 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26778
W20260924 16:09:25.869513 26948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108)
I20260924 16:09:25.869558 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26854
I20260924 16:09:25.871402 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:25.870866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38355 (local address 127.0.0.1:52176)
0924 16:09:25.871182 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:25.871271 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:38355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":56,"threads_started":1}
W20260924 16:09:25.871574 26948 consensus_peers.cc:487] T 00000000000000000000000000000000 P 28dad19a05a64e679ad02673576b4874 -> Peer f9a82c37ab1d4f489802d207d40a4703 (127.13.8.126:38355): Couldn't send request to peer f9a82c37ab1d4f489802d207d40a4703. Status: Network error: Client connection negotiation failed: client connection to 127.13.8.126:38355: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 16:09:25.876899 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 26931
2026-09-24T16:09:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3220 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:09:25 dist-test-slave-vj6n krb5kdc[27075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:09:25 dist-test-slave-vj6n krb5kdc[27075](info): set up 2 sockets
Sep 24 16:09:25 dist-test-slave-vj6n krb5kdc[27075](info): commencing operation
krb5kdc: starting...
W20260924 16:09:26.931499 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:09:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:09:27.057385 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41107
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:27.169544 27088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:27.169792 27088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:27.169834 27088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:27.173681 27088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:27.173727 27088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:27.173744 27088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:27.173760 27088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:27.173775 27088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:27.178831 27088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:27.179788 27088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:27.180727 27088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:27.189342 27094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:27.189349 27093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:27.189357 27096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:27.189533 27088 server_base.cc:1034] running on GCE node
I20260924 16:09:27.190336 27088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:27.191591 27088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:27.192816 27088 hybrid_clock.cc:648] HybridClock initialized: now 1790266167192798 us; error 35 us; skew 500 ppm
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266167, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:27.195616 27088 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:27.196579 27088 webserver.cc:460] Webserver started at http://127.13.8.126:38563/ using document root <none> and password file <none>
I20260924 16:09:27.197141 27088 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:27.197237 27088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:27.197474 27088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:27.199601 27088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "8b556b5dabf04a52a1aaba4fe958444a"
format_stamp: "Formatted at 2026-09-24 16:09:27 on dist-test-slave-vj6n"
server_key: "e15d42fd3c25efe3769a71e6cab7b9fc"
server_key_iv: "5345eb188c88cfed3af451de110fd997"
server_key_version: "encryptionkey@0"
I20260924 16:09:27.200174 27088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "8b556b5dabf04a52a1aaba4fe958444a"
format_stamp: "Formatted at 2026-09-24 16:09:27 on dist-test-slave-vj6n"
server_key: "e15d42fd3c25efe3769a71e6cab7b9fc"
server_key_iv: "5345eb188c88cfed3af451de110fd997"
server_key_version: "encryptionkey@0"
I20260924 16:09:27.204106 27088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:09:27.206876 27103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:27.208076 27088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:09:27.208228 27088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "8b556b5dabf04a52a1aaba4fe958444a"
format_stamp: "Formatted at 2026-09-24 16:09:27 on dist-test-slave-vj6n"
server_key: "e15d42fd3c25efe3769a71e6cab7b9fc"
server_key_iv: "5345eb188c88cfed3af451de110fd997"
server_key_version: "encryptionkey@0"
I20260924 16:09:27.208348 27088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:27.218606 27088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:27.221864 27088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:27.222012 27088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:27.228479 27088 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41107
I20260924 16:09:27.228577 27155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41107 every 8 connection(s)
I20260924 16:09:27.229460 27088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:27.232398 27156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:27.233651 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27088
I20260924 16:09:27.233785 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:09:27.234107 13345 external_mini_cluster.cc:1428] Setting key cb7768d7160fc5c95cb05bcce09d93d6
I20260924 16:09:27.239053 27156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a: Bootstrap starting.
I20260924 16:09:27.241796 27156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:27.242760 27156 log.cc:824] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:27.245263 27156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a: No bootstrap required, opened a new log
I20260924 16:09:27.249002 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:27.235428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.13.8.126:41107)
0924 16:09:27.235818 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:27.235834 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:27.235869 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:09:27.236511 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:27.236528 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:27.236858 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:27.237129 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:27.238011 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:27.239058 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:27.240229 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:27.240691 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:27.240803 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:27.243724 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:27.243732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:27.243749 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:27.243777 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:27.245871 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:27.246352 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:27.246357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:27.246363 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:27.246503 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:27.246784 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:27.246789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:27.246792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:27.247297 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:27.247517 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:27.247700 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 16:09:27.248041 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":120,"threads_started":1}
I20260924 16:09:27.249954 27156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } }
I20260924 16:09:27.250267 27156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:27.250330 27156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b556b5dabf04a52a1aaba4fe958444a, State: Initialized, Role: FOLLOWER
I20260924 16:09:27.250757 27156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } }
I20260924 16:09:27.250905 27156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:27.250988 27156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:27.251091 27156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:27.252151 27156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } }
I20260924 16:09:27.252331 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:09:27.252542 27156 leader_election.cc:302] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b556b5dabf04a52a1aaba4fe958444a; no voters:
I20260924 16:09:27.252890 27156 leader_election.cc:288] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:09:27.253003 27161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:27.253268 27161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [term 1 LEADER]: Becoming Leader. State: Replica: 8b556b5dabf04a52a1aaba4fe958444a, State: Running, Role: LEADER
I20260924 16:09:27.253635 27161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } }
I20260924 16:09:27.254343 27156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:27.255455 27162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b556b5dabf04a52a1aaba4fe958444a. Latest consensus state: current_term: 1 leader_uuid: "8b556b5dabf04a52a1aaba4fe958444a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } } }
I20260924 16:09:27.255450 27163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b556b5dabf04a52a1aaba4fe958444a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b556b5dabf04a52a1aaba4fe958444a" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41107 } } }
I20260924 16:09:27.255600 27162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:27.255604 27163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:27.256047 27170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:27.258877 27170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:27.264526 27170 catalog_manager.cc:1347] Generated new cluster ID: afbcd387b892418ebf0e621f7b427aaa
I20260924 16:09:27.264591 27170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:27.286017 27170 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:09:27.287477 27170 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:27.297533 27170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b556b5dabf04a52a1aaba4fe958444a: Generated new TSK 0
I20260924 16:09:27.298451 27170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:09:27.308301 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27088
2026-09-24T16:09:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1430 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27183](info): set up 2 sockets
Sep 24 16:09:27 dist-test-slave-vj6n krb5kdc[27183](info): commencing operation
krb5kdc: starting...
W20260924 16:09:28.363237 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:09:28 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:09:28.485348 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:28.601202 27197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:28.601477 27197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:28.601560 27197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:28.605854 27197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:28.605919 27197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:28.605939 27197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:28.605993 27197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:28.606027 27197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:28.611563 27197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:28.612668 27197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:28.613714 27197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:28.622237 27205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:28.622193 27202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:28.622193 27203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:28.622376 27197 server_base.cc:1034] running on GCE node
I20260924 16:09:28.623099 27197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:28.624202 27197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:28.625418 27197 hybrid_clock.cc:648] HybridClock initialized: now 1790266168625391 us; error 47 us; skew 500 ppm
Sep 24 16:09:28 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266168, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:28.632866 27197 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:28.633857 27197 webserver.cc:460] Webserver started at http://127.13.8.126:33075/ using document root <none> and password file <none>
I20260924 16:09:28.634558 27197 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:28.634666 27197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:28.634923 27197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:28.637089 27197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "b66e29737fcf4f4c86266665109560f3"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "df5b3c2e793884589bcd549ec7dbaa9b"
server_key_iv: "085e3c16539a9a31ecfed15e35246035"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.637672 27197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "b66e29737fcf4f4c86266665109560f3"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "df5b3c2e793884589bcd549ec7dbaa9b"
server_key_iv: "085e3c16539a9a31ecfed15e35246035"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.641865 27197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260924 16:09:28.644435 27212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:28.645642 27197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:09:28.645787 27197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "b66e29737fcf4f4c86266665109560f3"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "df5b3c2e793884589bcd549ec7dbaa9b"
server_key_iv: "085e3c16539a9a31ecfed15e35246035"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.645893 27197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:28.661839 27197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:28.664820 27197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:28.665035 27197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:28.672829 27197 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35247
I20260924 16:09:28.673141 27264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35247 every 8 connection(s)
I20260924 16:09:28.673877 27197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:28.676802 27265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:28.681838 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27197
I20260924 16:09:28.681994 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:09:28.682313 13345 external_mini_cluster.cc:1428] Setting key f57116045312ae72b1e77eb4edf180b1
I20260924 16:09:28.682922 27265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap starting.
I20260924 16:09:28.685354 27265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:28.686331 27265 log.cc:824] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:28.688508 27265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: No bootstrap required, opened a new log
Sep 24 16:09:28 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:28.693130 27265 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:28.693383 27265 raft_consensus.cc:374] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:28.693485 27265 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Initialized, Role: FOLLOWER
I20260924 16:09:28.694002 27265 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:28.694177 27265 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:28.694279 27265 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:28.694401 27265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:28.695487 27265 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:28.695884 27265 leader_election.cc:302] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66e29737fcf4f4c86266665109560f3; no voters:
I20260924 16:09:28.696278 27265 leader_election.cc:288] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:09:28.696342 27270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:28.696558 27270 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 LEADER]: Becoming Leader. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Running, Role: LEADER
I20260924 16:09:28.696921 27270 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:28.697094 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:28.683713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48338 (local address 127.13.8.126:35247)
0924 16:09:28.684136 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:28.684163 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:28.684200 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:09:28.684842 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:28.684858 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:28.685186 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:28.685510 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:28.686353 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.687375 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.688486 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.689017 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.689143 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:28.691893 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:28.691899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:28.691911 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:28.691938 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:28.693730 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:28.694319 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:28.694324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:28.694329 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:28.694508 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:28.694815 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:28.694818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:28.694819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:28.695209 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:28.695348 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:28.695759 (+ 411us) server_negotiation.cc:299] Negotiation successful
0924 16:09:28.696089 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":150,"threads_started":1}
I20260924 16:09:28.697650 27265 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:28.698161 27272 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66e29737fcf4f4c86266665109560f3. Latest consensus state: current_term: 1 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:28.698299 27272 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:28.699517 27275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:28.700508 27271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:28.700906 27271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:28.702739 27275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:28.708236 27275 catalog_manager.cc:1347] Generated new cluster ID: 7fb094e97dd94738b8bbf6ffbfe693f4
I20260924 16:09:28.708309 27275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:28.720427 27275 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:09:28.721750 27275 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:28.728157 27275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Generated new TSK 0
I20260924 16:09:28.728956 27275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:09:28.762130 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:28.874898 27291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:28.875192 27291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:28.875265 27291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:28.879244 27291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:28.879331 27291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:28.879418 27291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:28.884560 27291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:28.885643 27291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:28.886775 27291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:09:28.896521 27291 server_base.cc:1034] running on GCE node
W20260924 16:09:28.896461 27299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:28.896471 27297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:28.896481 27296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:28.897140 27291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:28.897826 27291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:28.899024 27291 hybrid_clock.cc:648] HybridClock initialized: now 1790266168898992 us; error 42 us; skew 500 ppm
Sep 24 16:09:28 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266168, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:28.901748 27291 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:28.902657 27291 webserver.cc:460] Webserver started at http://127.13.8.65:40777/ using document root <none> and password file <none>
I20260924 16:09:28.903169 27291 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:28.903226 27291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:28.903390 27291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:28.905393 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "3e2e8173142f4fe6bef28bec93db39a2"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "968ed30e529c0c714504fcab8e73c2de"
server_key_iv: "cc11950bed790c817232656f8eccc1a6"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.905901 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "3e2e8173142f4fe6bef28bec93db39a2"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "968ed30e529c0c714504fcab8e73c2de"
server_key_iv: "cc11950bed790c817232656f8eccc1a6"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.909462 27291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:09:28.911840 27306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:28.913058 27291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:28.913179 27291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "3e2e8173142f4fe6bef28bec93db39a2"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "968ed30e529c0c714504fcab8e73c2de"
server_key_iv: "cc11950bed790c817232656f8eccc1a6"
server_key_version: "encryptionkey@0"
I20260924 16:09:28.913276 27291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:28.927408 27291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:28.930434 27291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:28.930684 27291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:28.931284 27291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:28.932121 27291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:28.932168 27291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:28.932243 27291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:28.932276 27291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:28.940804 27291 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:35387
I20260924 16:09:28.940893 27420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:35387 every 8 connection(s)
I20260924 16:09:28.941715 27291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:28.948586 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27291
I20260924 16:09:28.948709 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:09:28.949025 13345 external_mini_cluster.cc:1428] Setting key bca4f92478b6265b6f2ed681a459e8f4
Sep 24 16:09:28 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266168, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:28.956634 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:28.943571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:32961 (local address 127.13.8.126:35247)
0924 16:09:28.943741 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:28.943747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:28.944582 (+ 835us) server_negotiation.cc:407] Connection header received
0924 16:09:28.945855 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:28.945857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:28.945901 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:28.945972 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:28.947407 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.947938 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.949468 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.949814 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.949906 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:28.952604 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:28.952609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:28.952613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:28.952646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:28.954182 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:28.954895 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:28.954897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:28.954899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:28.954981 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:28.955380 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:28.955386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:28.955391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:28.955660 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:28.955778 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:28.956271 (+ 493us) server_negotiation.cc:299] Negotiation successful
0924 16:09:28.956415 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 16:09:28.957329 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:28.943973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:32961)
0924 16:09:28.944423 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:28.944476 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:28.945669 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:28.946138 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:28.946144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:28.946639 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:28.947240 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:28.947258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.948065 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.948068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:28.949327 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:28.949338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:28.949960 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:28.949964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:28.950150 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:28.950811 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:28.950828 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:28.952439 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:28.954357 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:28.954366 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:28.954386 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:28.954784 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:28.955079 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:28.955084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:28.955087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:28.955275 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:28.955774 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:28.955779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:28.956006 (+ 227us) client_negotiation.cc:769] Sending connection context
0924 16:09:28.956177 (+ 171us) client_negotiation.cc:240] Negotiation successful
0924 16:09:28.956425 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260924 16:09:28.958647 27421 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:09:28.958897 27421 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:28.959368 27421 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
I20260924 16:09:28.960865 27229 ts_manager.cc:194] Registered new tserver with Master: 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65:35387)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:09:28.962095 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:32961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:09:28.977584 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:29.093870 27426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:29.094174 27426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:29.094303 27426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:29.098997 27426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:29.099086 27426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:29.099186 27426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:29.104911 27426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:29.106051 27426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:29.107230 27426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:29.117790 27434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.117837 27432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.117969 27431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:29.118409 27426 server_base.cc:1034] running on GCE node
I20260924 16:09:29.118851 27426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:29.119508 27426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:29.120699 27426 hybrid_clock.cc:648] HybridClock initialized: now 1790266169120681 us; error 36 us; skew 500 ppm
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:29.123517 27426 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:29.124671 27426 webserver.cc:460] Webserver started at http://127.13.8.66:35595/ using document root <none> and password file <none>
I20260924 16:09:29.125277 27426 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:29.125372 27426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:29.125650 27426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:29.127812 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "e780221940044f909a4b7a5b9a6e293d"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "0addfe383f55940258e5c3864409b7a4"
server_key_iv: "9031424ac3795d68b866dbf5d145822f"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.128381 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "e780221940044f909a4b7a5b9a6e293d"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "0addfe383f55940258e5c3864409b7a4"
server_key_iv: "9031424ac3795d68b866dbf5d145822f"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.132269 27426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:09:29.135007 27441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.136329 27426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:09:29.136471 27426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "e780221940044f909a4b7a5b9a6e293d"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "0addfe383f55940258e5c3864409b7a4"
server_key_iv: "9031424ac3795d68b866dbf5d145822f"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.136597 27426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:29.162367 27426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:29.165377 27426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:29.165593 27426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:29.166198 27426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:29.167114 27426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:29.167184 27426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.167268 27426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:29.167308 27426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.176673 27426 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38687
I20260924 16:09:29.176702 27555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38687 every 8 connection(s)
I20260924 16:09:29.177640 27426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:29.184161 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27426
I20260924 16:09:29.184320 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:09:29.184582 13345 external_mini_cluster.cc:1428] Setting key 20f7d412157fbe2872cfe9ac6e239d8e
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:29.192180 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.179569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:35679 (local address 127.13.8.126:35247)
0924 16:09:29.179773 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.179777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.180640 (+ 863us) server_negotiation.cc:407] Connection header received
0924 16:09:29.182079 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.182082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.182138 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.182240 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:29.183475 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.184072 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.185359 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.185591 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.185698 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.188448 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:29.188452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:29.188455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:29.188504 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:29.190069 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.190613 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.190616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.190618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.190711 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.191050 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.191053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.191055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.191240 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:29.191381 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.191902 (+ 521us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.192021 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260924 16:09:29.193112 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.179970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:35679)
0924 16:09:29.180481 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.180524 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.181855 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.182449 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.182456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.182826 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:29.183316 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.183331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.184207 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.184215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.185223 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.185234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.185752 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.185756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.185917 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.186560 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:29.186576 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:29.188283 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:29.190185 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.190190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.190206 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.190498 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.190809 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.190812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.190814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.190952 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.191391 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:29.191401 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:29.191659 (+ 258us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.191844 (+ 185us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.192179 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:09:29.194330 27556 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
I20260924 16:09:29.194630 27556 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:29.195235 27556 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:09:29.196583 27229 ts_manager.cc:194] Registered new tserver with Master: e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:29.197412 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:35679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:09:29.214035 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:29.325136 27561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:29.325369 27561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:29.325409 27561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:29.329246 27561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:29.329291 27561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:29.329345 27561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:29.334528 27561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:29.335491 27561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:29.336467 27561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:29.346418 27569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.346447 27567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.346419 27566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:29.346562 27561 server_base.cc:1034] running on GCE node
I20260924 16:09:29.347113 27561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:29.347754 27561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:29.348938 27561 hybrid_clock.cc:648] HybridClock initialized: now 1790266169348900 us; error 44 us; skew 500 ppm
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:29.351689 27561 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:29.352522 27561 webserver.cc:460] Webserver started at http://127.13.8.67:41517/ using document root <none> and password file <none>
I20260924 16:09:29.353058 27561 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:29.353127 27561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:29.353363 27561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:29.355410 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "6123643ed5594297b8b0fa157e9ca807"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "8a44395f4bd67ab2df99df96181ccab6"
server_key_iv: "22311eb53d2c45ae7487b7f97c59208d"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.355958 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "6123643ed5594297b8b0fa157e9ca807"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "8a44395f4bd67ab2df99df96181ccab6"
server_key_iv: "22311eb53d2c45ae7487b7f97c59208d"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.359573 27561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:09:29.361931 27576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.363186 27561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:09:29.363302 27561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "6123643ed5594297b8b0fa157e9ca807"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "8a44395f4bd67ab2df99df96181ccab6"
server_key_iv: "22311eb53d2c45ae7487b7f97c59208d"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.363404 27561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:29.380018 27561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:29.382817 27561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:29.383009 27561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:29.383606 27561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:29.384395 27561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:29.384438 27561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.384476 27561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:29.384522 27561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.393373 27561 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43565
I20260924 16:09:29.393402 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43565 every 8 connection(s)
I20260924 16:09:29.394315 27561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:09:29.399875 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27561
I20260924 16:09:29.400014 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:09:29.400298 13345 external_mini_cluster.cc:1428] Setting key a06e137561fc5098f5b3f5bc3236e09c
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:29.407833 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.396116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37613 (local address 127.13.8.126:35247)
0924 16:09:29.396299 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.396305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.397100 (+ 795us) server_negotiation.cc:407] Connection header received
0924 16:09:29.398572 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.398577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.398654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.398740 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:29.399976 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.400712 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.402056 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.402382 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.402478 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.404781 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:29.404783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:29.404785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:29.404809 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:29.406102 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.406604 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.406607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.406608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.406678 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.406958 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.406960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.406962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.407117 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:29.407188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.407630 (+ 442us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.407718 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:09:29.408623 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.396483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:37613)
0924 16:09:29.396942 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.396988 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.398322 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.398903 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.398909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.399318 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:29.399830 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.399843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.400844 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.400852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.401931 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.401941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.402526 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.402531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.402712 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.403200 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:29.403215 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:29.404637 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:29.406199 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.406204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.406266 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.406512 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.406756 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.406758 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.406759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.406883 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.407220 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:29.407224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:29.407439 (+ 215us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.407585 (+ 146us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.407817 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260924 16:09:29.409644 27691 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
I20260924 16:09:29.409868 27691 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:29.410352 27691 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
I20260924 16:09:29.411419 27229 ts_manager.cc:194] Registered new tserver with Master: 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565)
I20260924 16:09:29.411916 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:37613
I20260924 16:09:29.414120 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:09:29.423027 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.416065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.13.8.126:35247)
0924 16:09:29.416203 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.416206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.416397 (+ 191us) server_negotiation.cc:407] Connection header received
0924 16:09:29.416482 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.416484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.416518 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.416580 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:29.417263 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.417731 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.418597 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.418796 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.418873 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.420178 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:29.420181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:29.420182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:29.420203 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:29.421436 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.421859 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.421861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.421863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.421931 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.422184 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.422186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.422187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.422425 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:29.422529 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.422750 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.422834 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 16:09:29.425675 27229 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:09:29.428731 27229 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:09:29.443727 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.13.8.67:43565)
0924 16:09:29.438334 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.438347 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.438572 (+ 225us) server_negotiation.cc:407] Connection header received
0924 16:09:29.438665 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.438671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.438859 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.439003 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.439223 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440375 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442083 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.442725 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442846 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443012 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.443460 (+ 448us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.443565 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":99,"threads_started":1}
I20260924 16:09:29.443717 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.0.0.1:47636)
0924 16:09:29.438511 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.438525 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.438604 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.439266 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.439269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.439285 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.439365 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.439371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440375 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.440381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.441841 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.441852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.442959 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.443071 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443307 (+ 236us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.443407 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.443556 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":206,"thread_start_us":87,"threads_started":1}
I20260924 16:09:29.443725 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43565 (local address 127.0.0.1:41622)
0924 16:09:29.438366 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.438380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.438469 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.439006 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.439010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.439026 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.439107 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.439112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440506 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.440509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.441971 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.441978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.443068 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.443071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.443172 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443318 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.443424 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.443560 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":448,"thread_start_us":86,"threads_started":1}
I20260924 16:09:29.443764 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.13.8.65:35387)
0924 16:09:29.438200 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.438240 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.438267 (+ 27us) server_negotiation.cc:407] Connection header received
0924 16:09:29.438406 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.438412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.438624 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.438784 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.439079 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440295 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442033 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.442841 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442948 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443074 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.443539 (+ 465us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.443632 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":86,"threads_started":1}
I20260924 16:09:29.444070 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:35387 (local address 127.0.0.1:46804)
0924 16:09:29.437729 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.437777 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.437898 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.438782 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.438787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.438806 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.438950 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.438961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440418 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.440421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.441879 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.441888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.443123 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.443126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.443224 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443302 (+ 78us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.443407 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.443950 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":582,"thread_start_us":78,"threads_started":1}
I20260924 16:09:29.444468 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.437939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.13.8.66:38687)
0924 16:09:29.438861 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.438875 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.438896 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:09:29.438971 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.438978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.439145 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.439290 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.439443 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.440249 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.441973 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.442846 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.442945 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.443074 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.444092 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.444319 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":146,"threads_started":1}
I20260924 16:09:29.446904 27355 tablet_service.cc:1467] Processing CreateTablet for tablet 734b97a473ea4f13a17f5e958c22653e (DEFAULT_TABLE table=test-table [id=eddd82a21c144aab8ea4d35aadbe7843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:29.447046 27489 tablet_service.cc:1467] Processing CreateTablet for tablet 734b97a473ea4f13a17f5e958c22653e (DEFAULT_TABLE table=test-table [id=eddd82a21c144aab8ea4d35aadbe7843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:29.447160 27625 tablet_service.cc:1467] Processing CreateTablet for tablet 734b97a473ea4f13a17f5e958c22653e (DEFAULT_TABLE table=test-table [id=eddd82a21c144aab8ea4d35aadbe7843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:29.447994 27355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734b97a473ea4f13a17f5e958c22653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:29.448055 27625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734b97a473ea4f13a17f5e958c22653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:29.448048 27489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734b97a473ea4f13a17f5e958c22653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:29.453992 27708 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap starting.
I20260924 16:09:29.455507 27709 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Bootstrap starting.
I20260924 16:09:29.456036 27710 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Bootstrap starting.
I20260924 16:09:29.456295 27708 tablet_bootstrap.cc:654] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:29.457377 27708 log.cc:824] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:29.458118 27709 tablet_bootstrap.cc:654] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:29.458622 27710 tablet_bootstrap.cc:654] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:29.459049 27709 log.cc:824] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:29.459601 27708 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: No bootstrap required, opened a new log
I20260924 16:09:29.459812 27708 ts_tablet_manager.cc:1397] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:09:29.459893 27710 log.cc:824] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:29.461040 27709 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: No bootstrap required, opened a new log
I20260924 16:09:29.461241 27709 ts_tablet_manager.cc:1397] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:09:29.461813 27710 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: No bootstrap required, opened a new log
I20260924 16:09:29.461985 27710 ts_tablet_manager.cc:1397] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260924 16:09:29.463667 27708 raft_consensus.cc:348] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.463963 27708 raft_consensus.cc:374] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:29.464049 27708 raft_consensus.cc:729] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Initialized, Role: FOLLOWER
I20260924 16:09:29.464551 27708 consensus_queue.cc:260] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.464985 27709 raft_consensus.cc:348] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.465214 27421 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
I20260924 16:09:29.465241 27709 raft_consensus.cc:374] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:29.465339 27709 raft_consensus.cc:729] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e780221940044f909a4b7a5b9a6e293d, State: Initialized, Role: FOLLOWER
I20260924 16:09:29.465603 27708 ts_tablet_manager.cc:1428] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:09:29.465785 27709 consensus_queue.cc:260] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.466426 27556 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
I20260924 16:09:29.466713 27709 ts_tablet_manager.cc:1428] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 16:09:29.467415 27710 raft_consensus.cc:348] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.467669 27710 raft_consensus.cc:374] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:29.467765 27710 raft_consensus.cc:729] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6123643ed5594297b8b0fa157e9ca807, State: Initialized, Role: FOLLOWER
I20260924 16:09:29.468201 27710 consensus_queue.cc:260] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.468891 27691 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
I20260924 16:09:29.469172 27710 ts_tablet_manager.cc:1428] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260924 16:09:29.470054 27714 raft_consensus.cc:482] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:29.470296 27714 raft_consensus.cc:504] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.471451 27714 leader_election.cc:288] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:29.475348 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.472003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:35439 (local address 127.13.8.67:43565)
0924 16:09:29.472229 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.472234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.472248 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:09:29.472292 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.472304 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.472355 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.472424 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.472805 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.473327 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.474415 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.474811 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.474931 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.474990 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.475174 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.475238 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260924 16:09:29.475364 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.471628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43565 (local address 127.13.8.65:35439)
0924 16:09:29.471924 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.471943 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.472053 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.472540 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.472543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.472563 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.472688 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.472693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.473438 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.473443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.474292 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.474303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.474943 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.474947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.475039 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.475061 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.475159 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.475228 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":129,"threads_started":1}
I20260924 16:09:29.475843 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.472088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:39161 (local address 127.13.8.66:38687)
0924 16:09:29.472234 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.472239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.472361 (+ 122us) server_negotiation.cc:407] Connection header received
0924 16:09:29.472525 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.472528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.472581 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.472662 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.472963 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.473465 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.474591 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.475121 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.475227 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.475294 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.475664 (+ 370us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.475745 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:09:29.476184 27645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734b97a473ea4f13a17f5e958c22653e" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6123643ed5594297b8b0fa157e9ca807" is_pre_election: true
I20260924 16:09:29.476328 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.471694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.13.8.65:39161)
0924 16:09:29.472261 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.472274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.472413 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.472689 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.472692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.472710 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.472841 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.472847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.473594 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.473598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.474461 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.474472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.475362 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.475367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.475493 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.475529 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.476064 (+ 535us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.476164 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":117,"threads_started":1}
I20260924 16:09:29.476486 27645 raft_consensus.cc:2454] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 0.
I20260924 16:09:29.476953 27310 leader_election.cc:302] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2, 6123643ed5594297b8b0fa157e9ca807; no voters:
I20260924 16:09:29.476958 27510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734b97a473ea4f13a17f5e958c22653e" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e780221940044f909a4b7a5b9a6e293d" is_pre_election: true
I20260924 16:09:29.477196 27510 raft_consensus.cc:2454] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 0.
I20260924 16:09:29.477206 27714 raft_consensus.cc:2781] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:09:29.477310 27714 raft_consensus.cc:482] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:09:29.477375 27714 raft_consensus.cc:3037] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:29.478608 27714 raft_consensus.cc:504] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.479015 27714 leader_election.cc:288] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 election: Requested vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:29.479292 27645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734b97a473ea4f13a17f5e958c22653e" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6123643ed5594297b8b0fa157e9ca807"
I20260924 16:09:29.479370 27510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734b97a473ea4f13a17f5e958c22653e" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e780221940044f909a4b7a5b9a6e293d"
I20260924 16:09:29.479456 27645 raft_consensus.cc:3037] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:29.479475 27510 raft_consensus.cc:3037] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:29.480427 27510 raft_consensus.cc:2454] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 1.
I20260924 16:09:29.480441 27645 raft_consensus.cc:2454] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 1.
I20260924 16:09:29.480693 27309 leader_election.cc:302] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2, e780221940044f909a4b7a5b9a6e293d; no voters:
I20260924 16:09:29.480852 27714 raft_consensus.cc:2781] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:29.481043 27714 raft_consensus.cc:686] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 LEADER]: Becoming Leader. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Running, Role: LEADER
I20260924 16:09:29.481328 27714 consensus_queue.cc:237] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.484532 27227 catalog_manager.cc:5495] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65). New cstate: current_term: 1 leader_uuid: "3e2e8173142f4fe6bef28bec93db39a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } health_report { overall_health: UNKNOWN } } }
I20260924 16:09:29.502282 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.498761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.13.8.65:35387)
0924 16:09:29.498890 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.498894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.498906 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:09:29.498991 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.498994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.499049 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.499139 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:09:29.499347 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.499956 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.500749 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.500951 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.501032 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.501357 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:09:29.501811 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:09:29.501958 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.501993 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.502058 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:09:29.508790 27645 raft_consensus.cc:1261] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Refusing update from remote peer 3e2e8173142f4fe6bef28bec93db39a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:29.508790 27510 raft_consensus.cc:1261] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Refusing update from remote peer 3e2e8173142f4fe6bef28bec93db39a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:29.509594 27719 consensus_queue.cc:1035] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:29.509795 27714 consensus_queue.cc:1035] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:29.518720 27724 mvcc.cc:205] Tried to move back new op lower bound from 7332930230300008448 to 7332930230197997568. Current Snapshot: MvccSnapshot[applied={T|T < 7332930230300008448}]
I20260924 16:09:29.527928 27229 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:29.528113 27229 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:29.530532 27229 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Sending DeleteTablet for 3 replicas of tablet 734b97a473ea4f13a17f5e958c22653e
I20260924 16:09:29.531596 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27291
I20260924 16:09:29.531608 27355 tablet_service.cc:1514] Processing DeleteTablet for tablet 734b97a473ea4f13a17f5e958c22653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:29 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:46804
I20260924 16:09:29.531616 27625 tablet_service.cc:1514] Processing DeleteTablet for tablet 734b97a473ea4f13a17f5e958c22653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:29 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41622
I20260924 16:09:29.532892 27734 tablet_replica.cc:330] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: stopping tablet replica
I20260924 16:09:29.533401 27734 raft_consensus.cc:2229] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:29.533792 27734 raft_consensus.cc:2258] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:29.534555 27489 tablet_service.cc:1514] Processing DeleteTablet for tablet 734b97a473ea4f13a17f5e958c22653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:29 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:47636
I20260924 16:09:29.535071 27735 tablet_replica.cc:330] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: stopping tablet replica
I20260924 16:09:29.535413 27735 raft_consensus.cc:2229] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:29.535557 27734 ts_tablet_manager.cc:1905] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:29.535724 27735 raft_consensus.cc:2258] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:29.537791 27734 ts_tablet_manager.cc:1918] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:29.537909 27734 log.cc:1196] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/734b97a473ea4f13a17f5e958c22653e
I20260924 16:09:29.538301 27734 ts_tablet_manager.cc:1939] T 734b97a473ea4f13a17f5e958c22653e P 6123643ed5594297b8b0fa157e9ca807: Deleting consensus metadata
I20260924 16:09:29.539275 27735 ts_tablet_manager.cc:1905] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:29.539417 27216 catalog_manager.cc:4841] TS 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): tablet 734b97a473ea4f13a17f5e958c22653e (table test-table [id=eddd82a21c144aab8ea4d35aadbe7843]) successfully deleted
I20260924 16:09:29.540979 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27426
W20260924 16:09:29.541020 27215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:35387 (error 108)
I20260924 16:09:29.542554 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.542185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:35387 (local address 127.0.0.1:46826)
0924 16:09:29.542363 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.542442 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:35387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260924 16:09:29.542783 27215 catalog_manager.cc:4584] TS 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65:35387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 734b97a473ea4f13a17f5e958c22653e: Network error: Client connection negotiation failed: client connection to 127.13.8.65:35387: connect: Connection refused (error 111)
I20260924 16:09:29.548450 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27561
I20260924 16:09:29.549520 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.548610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.0.0.1:47644)
0924 16:09:29.549363 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.549424 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.66:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657}
I20260924 16:09:29.554319 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27197
I20260924 16:09:29.560878 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--webserver_interface=127.13.8.126
--webserver_port=33075
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:29.672348 27737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:29.672612 27737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:29.672652 27737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:29.676702 27737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:29.676759 27737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:29.676777 27737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:29.676795 27737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:29.676846 27737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:29.682027 27737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=33075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:29.683310 27737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:29.684316 27737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:29.697652 27743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.697662 27744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:29.697666 27737 server_base.cc:1034] running on GCE node
W20260924 16:09:29.697713 27746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:29.698554 27737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:29.699659 27737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:29.700830 27737 hybrid_clock.cc:648] HybridClock initialized: now 1790266169700816 us; error 34 us; skew 500 ppm
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:29.703776 27737 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:29.704788 27737 webserver.cc:460] Webserver started at http://127.13.8.126:33075/ using document root <none> and password file <none>
I20260924 16:09:29.705411 27737 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:29.705487 27737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:29.709673 27737 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:29.712051 27753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.713405 27737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:09:29.713517 27737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "b66e29737fcf4f4c86266665109560f3"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "df5b3c2e793884589bcd549ec7dbaa9b"
server_key_iv: "085e3c16539a9a31ecfed15e35246035"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.714202 27737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:29.748204 27737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:29.751438 27737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:29.751658 27737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:29.759358 27737 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35247
I20260924 16:09:29.759671 27805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35247 every 8 connection(s)
I20260924 16:09:29.760545 27737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:29.765241 27806 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:09:29.767045 27806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap starting.
I20260924 16:09:29.767539 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27737
I20260924 16:09:29.768049 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:35387
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=40777
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:29.785907 27806 log.cc:824] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:29.795073 27806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:29.795481 27806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap complete.
I20260924 16:09:29.799890 27806 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:29.800316 27806 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Initialized, Role: FOLLOWER
I20260924 16:09:29.800892 27806 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:29.801054 27806 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:29.801167 27806 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:29.801301 27806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:09:29.803488 27806 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:29.803831 27806 leader_election.cc:302] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66e29737fcf4f4c86266665109560f3; no voters:
I20260924 16:09:29.804205 27806 leader_election.cc:288] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 16:09:29.804270 27810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 16:09:29.804517 27810 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 LEADER]: Becoming Leader. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Running, Role: LEADER
I20260924 16:09:29.804876 27810 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:29.805550 27806 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:29.805996 27810 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66e29737fcf4f4c86266665109560f3. Latest consensus state: current_term: 2 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:29.806110 27810 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:29.806583 27811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:29.806758 27811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:29.807077 27814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:29.812680 27814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:29.813318 27814 catalog_manager.cc:1259] Loaded cluster ID: 7fb094e97dd94738b8bbf6ffbfe693f4
I20260924 16:09:29.813390 27814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:29.815146 27814 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 16:09:29.819619 27814 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 16:09:29.825155 27814 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:29.825982 27814 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Loaded TSK: 0
I20260924 16:09:29.827313 27814 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:09:29.890857 27808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:29.891095 27808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:29.891139 27808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:29.895269 27808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:29.895334 27808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:29.895406 27808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:29.900491 27808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:35387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=40777
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:29.901507 27808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:29.902563 27808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:29.912750 27836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.912750 27834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:29.912829 27833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:29.913206 27808 server_base.cc:1034] running on GCE node
I20260924 16:09:29.913619 27808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:29.914352 27808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:29.915534 27808 hybrid_clock.cc:648] HybridClock initialized: now 1790266169915486 us; error 87 us; skew 500 ppm
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:29.918599 27808 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:29.919584 27808 webserver.cc:460] Webserver started at http://127.13.8.65:40777/ using document root <none> and password file <none>
I20260924 16:09:29.920154 27808 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:29.920279 27808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:29.924331 27808 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:09:29.926561 27843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:29.927848 27808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:09:29.927968 27808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "3e2e8173142f4fe6bef28bec93db39a2"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "968ed30e529c0c714504fcab8e73c2de"
server_key_iv: "cc11950bed790c817232656f8eccc1a6"
server_key_version: "encryptionkey@0"
I20260924 16:09:29.928580 27808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:29.941902 27808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:29.945017 27808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:29.945232 27808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:29.945869 27808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:29.947078 27850 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 16:09:29.951200 27808 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 16:09:29.951260 27808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260924 16:09:29.951330 27808 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 16:09:29.952977 27808 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 16:09:29.953034 27808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260924 16:09:29.953307 27850 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap starting.
I20260924 16:09:29.964003 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:35387 every 8 connection(s)
I20260924 16:09:29.964001 27808 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:35387
I20260924 16:09:29.965287 27808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:29.973935 27850 log.cc:824] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:29.975099 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27808
I20260924 16:09:29.975569 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:38687
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=35595
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:09:29 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:29.982270 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.967608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:59905)
0924 16:09:29.968243 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.968284 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.969099 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.970197 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.970205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.970655 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:29.971122 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.971135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.972438 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.972442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.973310 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.973320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.974065 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.974069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.974325 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.975034 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:29.975051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:29.976617 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:29.979076 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.979082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.979095 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.979349 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.979715 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:29.979719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:29.979721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:29.979864 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:29.980484 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:29.980490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:29.980747 (+ 257us) client_negotiation.cc:769] Sending connection context
0924 16:09:29.980932 (+ 185us) client_negotiation.cc:240] Negotiation successful
0924 16:09:29.981266 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":150,"threads_started":1}
I20260924 16:09:29.982285 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.967685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59905 (local address 127.13.8.126:35247)
0924 16:09:29.968056 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.968077 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.968414 (+ 337us) server_negotiation.cc:407] Connection header received
0924 16:09:29.969516 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.969526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.969811 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.970138 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:29.971303 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.972255 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.973443 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.973922 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.974098 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.976815 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:29.976820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:29.976833 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:29.976861 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:29.978954 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.979480 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.979486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.979498 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.979615 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:29.980009 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:29.980012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:29.980014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:29.980354 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:29.980506 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:29.981059 (+ 553us) server_negotiation.cc:299] Negotiation successful
0924 16:09:29.981324 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
I20260924 16:09:29.983691 27850 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:29.984148 27850 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap complete.
I20260924 16:09:29.984790 27959 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
I20260924 16:09:29.985031 27959 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:29.985525 27959 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
I20260924 16:09:29.985774 27850 ts_tablet_manager.cc:1397] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.007s
I20260924 16:09:29.988349 27770 ts_manager.cc:194] Registered new tserver with Master: 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65:35387)
I20260924 16:09:29.989269 27850 raft_consensus.cc:348] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.989809 27850 raft_consensus.cc:729] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Initialized, Role: FOLLOWER
I20260924 16:09:29.990350 27850 consensus_queue.cc:260] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:29.991372 27850 ts_tablet_manager.cc:1428] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260924 16:09:29.992964 27770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:59905
I20260924 16:09:29.995472 27959 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
I20260924 16:09:30.000382 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.991925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.13.8.65:35387)
0924 16:09:29.992288 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:29.992302 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:29.995818 (+ 3516us) server_negotiation.cc:407] Connection header received
0924 16:09:29.995899 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:29.995906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:29.996110 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:29.996280 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:29.996585 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.997502 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.999149 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.999664 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.999751 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.999849 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.000093 (+ 244us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.000180 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260924 16:09:30.000387 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:29.991585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:35387 (local address 127.0.0.1:46834)
0924 16:09:29.993931 (+ 2346us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:29.993974 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:29.994112 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:29.996291 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:29.996297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:29.996325 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:29.996458 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.996464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.997616 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.997620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.999031 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:29.999043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:29.999784 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:29.999788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:29.999908 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:29.999949 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.000058 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.000232 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2176,"thread_start_us":97,"threads_started":1}
I20260924 16:09:30.001596 27893 tablet_service.cc:1514] Processing DeleteTablet for tablet 734b97a473ea4f13a17f5e958c22653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:29 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:46834
I20260924 16:09:30.002458 27968 tablet_replica.cc:330] stopping tablet replica
I20260924 16:09:30.002766 27968 raft_consensus.cc:2229] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:30.003022 27968 raft_consensus.cc:2258] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:30.004233 27968 ts_tablet_manager.cc:1905] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:30.008225 27968 ts_tablet_manager.cc:1918] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:30.008311 27968 log.cc:1196] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/734b97a473ea4f13a17f5e958c22653e
I20260924 16:09:30.008605 27968 ts_tablet_manager.cc:1939] T 734b97a473ea4f13a17f5e958c22653e P 3e2e8173142f4fe6bef28bec93db39a2: Deleting consensus metadata
I20260924 16:09:30.010425 27756 catalog_manager.cc:4841] TS 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65:35387): tablet 734b97a473ea4f13a17f5e958c22653e (table test-table [id=eddd82a21c144aab8ea4d35aadbe7843]) successfully deleted
W20260924 16:09:30.094822 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:30.095075 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:30.095126 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:30.099134 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:30.099185 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:30.099242 27964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:30.104539 27964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:38687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=35595
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:30.105660 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:30.106822 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:30.116730 27973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:30.116816 27964 server_base.cc:1034] running on GCE node
W20260924 16:09:30.116731 27976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:30.116770 27974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:30.117375 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:30.118121 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:30.119297 27964 hybrid_clock.cc:648] HybridClock initialized: now 1790266170119283 us; error 34 us; skew 500 ppm
Sep 24 16:09:30 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266170, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:30.122045 27964 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:30.123005 27964 webserver.cc:460] Webserver started at http://127.13.8.66:35595/ using document root <none> and password file <none>
I20260924 16:09:30.123539 27964 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:30.123620 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:30.127336 27964 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 16:09:30.129560 27983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:30.130857 27964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260924 16:09:30.130975 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "e780221940044f909a4b7a5b9a6e293d"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "0addfe383f55940258e5c3864409b7a4"
server_key_iv: "9031424ac3795d68b866dbf5d145822f"
server_key_version: "encryptionkey@0"
I20260924 16:09:30.131569 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:30.140779 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:30.143579 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:30.143779 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:30.144371 27964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:30.145406 27990 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 16:09:30.149495 27964 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 16:09:30.149564 27964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260924 16:09:30.149634 27964 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 16:09:30.151355 27964 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 16:09:30.151397 27964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:30.151710 27990 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Bootstrap starting.
I20260924 16:09:30.157224 27990 log.cc:824] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:30.161883 27964 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38687
I20260924 16:09:30.161954 28098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38687 every 8 connection(s)
I20260924 16:09:30.162871 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:30.166556 27990 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:30.166922 27990 tablet_bootstrap.cc:492] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Bootstrap complete.
I20260924 16:09:30.167228 27990 ts_tablet_manager.cc:1397] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260924 16:09:30.169364 27990 raft_consensus.cc:348] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:30.169767 27990 raft_consensus.cc:729] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e780221940044f909a4b7a5b9a6e293d, State: Initialized, Role: FOLLOWER
I20260924 16:09:30.170289 27990 consensus_queue.cc:260] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } } peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } }
I20260924 16:09:30.171241 27990 ts_tablet_manager.cc:1428] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
Sep 24 16:09:30 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266170, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:30.173045 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 27964
I20260924 16:09:30.173475 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:43565
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=41517
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:30.176311 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.164686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53741 (local address 127.13.8.126:35247)
0924 16:09:30.164816 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.164820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.165426 (+ 606us) server_negotiation.cc:407] Connection header received
0924 16:09:30.166467 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.166471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.166550 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.166643 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:30.167872 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.168394 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.169478 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.169722 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.169841 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.172698 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:30.172701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:30.172703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:30.172729 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:30.174276 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.174786 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.174789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.174791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.174876 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.175227 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.175232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.175234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.175410 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:30.175516 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.175975 (+ 459us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.176103 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:09:30.177002 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.164810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:53741)
0924 16:09:30.165277 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.165317 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.166194 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.166843 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.166849 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.167229 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:30.167727 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.167739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.168546 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.168549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.169347 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.169356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.169873 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.169877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.170056 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.171134 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:30.171150 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:30.172538 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:30.174398 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.174403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.174416 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.174674 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.174980 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.174985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.174986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.175131 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.175513 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:30.175518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:30.175750 (+ 232us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.175920 (+ 170us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.176263 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":136,"threads_started":1}
I20260924 16:09:30.178184 28099 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
I20260924 16:09:30.178442 28099 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:30.178930 28099 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
I20260924 16:09:30.180584 27770 ts_manager.cc:194] Registered new tserver with Master: e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:30.181764 27770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:53741
I20260924 16:09:30.183765 28099 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
Sep 24 16:09:30 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266169, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.66@KRBTEST.COM
I20260924 16:09:30.191341 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.181415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.0.0.1:47646)
0924 16:09:30.181581 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.181604 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.181670 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.182336 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.182341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.182878 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:30.183015 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.183025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.184302 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.184308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.184833 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.184841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.185545 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.185549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.185644 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.186207 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:30.186261 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:30.187736 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:30.189599 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.189609 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.189615 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.189911 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.190193 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.190196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.190199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.190315 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.190769 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:30.190776 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:30.190908 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.191031 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.191169 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260924 16:09:30.191362 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.181563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.13.8.66:38687)
0924 16:09:30.181895 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.181906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.181927 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:09:30.181987 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.181994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.182185 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.182334 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:30.183158 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.184178 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.184960 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.185411 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.185530 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.187939 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:30.187947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:30.187957 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:30.187977 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:30.189471 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.190026 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.190031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.190032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.190107 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.190407 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.190409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.190410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.190645 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:30.190749 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.191103 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.191231 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
I20260924 16:09:30.192538 28032 tablet_service.cc:1514] Processing DeleteTablet for tablet 734b97a473ea4f13a17f5e958c22653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:29 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:47646
I20260924 16:09:30.193437 28105 tablet_replica.cc:330] stopping tablet replica
I20260924 16:09:30.193810 28105 raft_consensus.cc:2229] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:30.194075 28105 raft_consensus.cc:2258] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:30.195240 28105 ts_tablet_manager.cc:1905] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:30.199944 28105 ts_tablet_manager.cc:1918] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:30.200042 28105 log.cc:1196] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/734b97a473ea4f13a17f5e958c22653e
I20260924 16:09:30.200342 28105 ts_tablet_manager.cc:1939] T 734b97a473ea4f13a17f5e958c22653e P e780221940044f909a4b7a5b9a6e293d: Deleting consensus metadata
I20260924 16:09:30.202270 27756 catalog_manager.cc:4841] TS e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): tablet 734b97a473ea4f13a17f5e958c22653e (table test-table [id=eddd82a21c144aab8ea4d35aadbe7843]) successfully deleted
W20260924 16:09:30.295575 28103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:30.295822 28103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:30.295866 28103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:30.299871 28103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:30.299921 28103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:30.299976 28103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:30.305131 28103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:43565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=41517
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:30.306162 28103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:30.307384 28103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:30.317226 28111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:30.317246 28113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:30.317274 28110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:30.317605 28103 server_base.cc:1034] running on GCE node
I20260924 16:09:30.318003 28103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:30.318727 28103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:30.319908 28103 hybrid_clock.cc:648] HybridClock initialized: now 1790266170319889 us; error 28 us; skew 500 ppm
Sep 24 16:09:30 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266170, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:30.322659 28103 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:30.323619 28103 webserver.cc:460] Webserver started at http://127.13.8.67:41517/ using document root <none> and password file <none>
I20260924 16:09:30.324231 28103 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:30.324330 28103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:30.328117 28103 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260924 16:09:30.330106 28120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:30.331368 28103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:09:30.331496 28103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "6123643ed5594297b8b0fa157e9ca807"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "8a44395f4bd67ab2df99df96181ccab6"
server_key_iv: "22311eb53d2c45ae7487b7f97c59208d"
server_key_version: "encryptionkey@0"
I20260924 16:09:30.332070 28103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:30.345659 28103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:30.348383 28103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:30.348589 28103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:30.349170 28103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:30.349977 28103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:30.350019 28103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:30.350092 28103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:30.350128 28103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:30.358896 28103 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43565
I20260924 16:09:30.358922 28233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43565 every 8 connection(s)
I20260924 16:09:30.359901 28103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
Sep 24 16:09:30 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266170, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:30.370054 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28103
I20260924 16:09:30.373817 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.361727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:49143 (local address 127.13.8.126:35247)
0924 16:09:30.361877 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.361881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.362775 (+ 894us) server_negotiation.cc:407] Connection header received
0924 16:09:30.364151 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.364155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.364215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.364326 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:30.365493 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.366011 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.367180 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.367403 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.367525 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.369994 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:30.369998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:30.370000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:30.370024 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:30.371623 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.372243 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.372248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.372250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.372335 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.372689 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.372693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.372695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.372900 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:30.373013 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.373510 (+ 497us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.373642 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":62}
I20260924 16:09:30.374653 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.362108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:49143)
0924 16:09:30.362628 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.362668 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.363921 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.364508 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.364514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.364859 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:30.365335 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.365348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.366135 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.366139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.367034 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.367044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.367562 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.367578 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.367742 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.368378 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:30.368393 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:30.369824 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:30.371728 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.371733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.371745 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.372154 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.372446 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:30.372449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:30.372451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:30.372590 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:30.373014 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:30.373022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:30.373267 (+ 245us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.373456 (+ 189us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.373778 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":126,"threads_started":1}
I20260924 16:09:30.375988 28234 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35247
I20260924 16:09:30.376271 28234 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:30.376770 28234 heartbeater.cc:511] Master 127.13.8.126:35247 requested a full tablet report, sending...
I20260924 16:09:30.377859 27770 ts_manager.cc:194] Registered new tserver with Master: 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565)
I20260924 16:09:30.378651 27770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:49143
I20260924 16:09:30.379704 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.370801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.13.8.126:35247)
0924 16:09:30.371134 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.371139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.371856 (+ 717us) server_negotiation.cc:407] Connection header received
0924 16:09:30.371948 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.371951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.372008 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.372107 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:30.372774 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.373327 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.374377 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.374616 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.374721 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.376316 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:30.376320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:30.376323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:30.376349 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:30.377994 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.378459 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.378461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.378462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.378534 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:30.378843 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:30.378846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:30.378847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:30.379065 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:30.379252 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.379441 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.379555 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
I20260924 16:09:30.381310 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:09:30.385548 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.381898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.13.8.126:35247)
0924 16:09:30.382036 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.382040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.382194 (+ 154us) server_negotiation.cc:407] Connection header received
0924 16:09:30.382386 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.382389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.382515 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.382622 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:09:30.382852 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.383337 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.384200 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.384417 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.384520 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.384829 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:09:30.385188 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:09:30.385312 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.385374 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.385426 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 16:09:30.386132 27770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:09:30.389380 27770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:09:30.396734 28032 tablet_service.cc:1467] Processing CreateTablet for tablet 1d44eca229f14baebdf78f71ef947c24 (DEFAULT_TABLE table=test-table [id=66aa85a89ecf41fdb6866f18a83fbcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:30.396734 27893 tablet_service.cc:1467] Processing CreateTablet for tablet 1d44eca229f14baebdf78f71ef947c24 (DEFAULT_TABLE table=test-table [id=66aa85a89ecf41fdb6866f18a83fbcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:30.397365 28032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d44eca229f14baebdf78f71ef947c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:30.397365 27893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d44eca229f14baebdf78f71ef947c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:30.400259 27990 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Bootstrap starting.
I20260924 16:09:30.400259 27850 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap starting.
I20260924 16:09:30.401182 27850 tablet_bootstrap.cc:654] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:30.401182 27990 tablet_bootstrap.cc:654] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:30.402972 27850 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: No bootstrap required, opened a new log
I20260924 16:09:30.403049 27990 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: No bootstrap required, opened a new log
I20260924 16:09:30.403067 27850 ts_tablet_manager.cc:1397] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:30.403128 27990 ts_tablet_manager.cc:1397] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:30.403530 27850 raft_consensus.cc:348] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.403530 27990 raft_consensus.cc:348] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.403656 27850 raft_consensus.cc:374] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:30.403656 27990 raft_consensus.cc:374] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:30.403714 27850 raft_consensus.cc:729] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Initialized, Role: FOLLOWER
I20260924 16:09:30.403714 27990 raft_consensus.cc:729] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e780221940044f909a4b7a5b9a6e293d, State: Initialized, Role: FOLLOWER
I20260924 16:09:30.403842 27850 consensus_queue.cc:260] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.403864 27990 consensus_queue.cc:260] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.404093 27990 ts_tablet_manager.cc:1428] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:09:30.404093 27850 ts_tablet_manager.cc:1428] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:09:30.404902 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.395848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43565 (local address 127.0.0.1:41630)
0924 16:09:30.396108 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.396135 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.396208 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.400978 (+ 4770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.400981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.400999 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:30.401126 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.401136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.402178 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.402185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.403354 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.403364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.404458 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.404462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.404554 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.404580 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.404679 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.404778 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260924 16:09:30.405120 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.396051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.13.8.67:43565)
0924 16:09:30.400475 (+ 4424us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.400493 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.400518 (+ 25us) server_negotiation.cc:407] Connection header received
0924 16:09:30.400614 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.400619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.400838 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.401011 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:30.401252 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.402035 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.403474 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.404328 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.404439 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.404603 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.404830 (+ 227us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.404934 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4303,"thread_start_us":167,"threads_started":1}
I20260924 16:09:30.408345 28168 tablet_service.cc:1467] Processing CreateTablet for tablet 1d44eca229f14baebdf78f71ef947c24 (DEFAULT_TABLE table=test-table [id=66aa85a89ecf41fdb6866f18a83fbcc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:30.409309 28168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d44eca229f14baebdf78f71ef947c24. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 16:09:30.413765 28100 tablet.cc:2369] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:09:30.416193 28246 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Bootstrap starting.
I20260924 16:09:30.418701 28246 tablet_bootstrap.cc:654] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:30.419371 28246 log.cc:824] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:30.421062 28246 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: No bootstrap required, opened a new log
I20260924 16:09:30.421219 28246 ts_tablet_manager.cc:1397] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:09:30.424273 28246 raft_consensus.cc:348] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.424479 28246 raft_consensus.cc:374] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:30.424520 28246 raft_consensus.cc:729] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6123643ed5594297b8b0fa157e9ca807, State: Initialized, Role: FOLLOWER
I20260924 16:09:30.424860 28246 consensus_queue.cc:260] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.425436 28234 heartbeater.cc:503] Master 127.13.8.126:35247 was elected leader, sending a full tablet report...
I20260924 16:09:30.425696 28246 ts_tablet_manager.cc:1428] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260924 16:09:30.466732 27960 tablet.cc:2369] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:09:30.610983 28235 tablet.cc:2369] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:09:30.808357 27965 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:30.808658 27965 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.810075 27965 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:30.813949 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.810405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36841 (local address 127.13.8.67:43565)
0924 16:09:30.810541 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.810547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.810738 (+ 191us) server_negotiation.cc:407] Connection header received
0924 16:09:30.810927 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.810930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.810982 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.811068 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:30.811331 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.811845 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.812983 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.813381 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.813500 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.813561 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.813740 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.813815 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:09:30.813957 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.810464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:45561 (local address 127.13.8.66:38687)
0924 16:09:30.810701 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.810706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.810807 (+ 101us) server_negotiation.cc:407] Connection header received
0924 16:09:30.810965 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.810968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.811032 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.811156 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:30.811398 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.811916 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.812966 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.813422 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.813543 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.813710 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.813747 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.813819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
I20260924 16:09:30.813961 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.810265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43565 (local address 127.13.8.65:36841)
0924 16:09:30.810629 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.810642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.810781 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.811081 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.811084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.811102 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:30.811215 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.811221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.811960 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.811963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.812846 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.812856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.813509 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.813513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.813603 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.813633 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.813722 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.813828 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":69,"thread_start_us":89,"threads_started":1}
I20260924 16:09:30.813975 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.810360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.13.8.65:45561)
0924 16:09:30.810705 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.810720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:30.810860 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:30.811168 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:30.811171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:30.811189 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:30.811289 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.811295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.812033 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.812037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.812844 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:30.812852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.813547 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.813551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:30.813640 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.813661 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 16:09:30.813726 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 16:09:30.813815 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":21,"thread_start_us":66,"threads_started":1}
I20260924 16:09:30.814687 28188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d44eca229f14baebdf78f71ef947c24" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6123643ed5594297b8b0fa157e9ca807" is_pre_election: true
I20260924 16:09:30.814762 28053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d44eca229f14baebdf78f71ef947c24" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e780221940044f909a4b7a5b9a6e293d" is_pre_election: true
I20260924 16:09:30.815032 28188 raft_consensus.cc:2454] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 0.
I20260924 16:09:30.815099 28053 raft_consensus.cc:2454] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 0.
I20260924 16:09:30.815527 27847 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2, 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d; no voters:
I20260924 16:09:30.815812 27965 raft_consensus.cc:2781] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:09:30.815902 27965 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:09:30.815946 27965 raft_consensus.cc:3037] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:30.816993 27965 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.817461 27965 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 election: Requested vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:30.817619 28188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d44eca229f14baebdf78f71ef947c24" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6123643ed5594297b8b0fa157e9ca807"
I20260924 16:09:30.817633 28053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d44eca229f14baebdf78f71ef947c24" candidate_uuid: "3e2e8173142f4fe6bef28bec93db39a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e780221940044f909a4b7a5b9a6e293d"
I20260924 16:09:30.817770 28053 raft_consensus.cc:3037] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:30.817759 28188 raft_consensus.cc:3037] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:30.819072 28053 raft_consensus.cc:2454] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 1.
I20260924 16:09:30.819072 28188 raft_consensus.cc:2454] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2e8173142f4fe6bef28bec93db39a2 in term 1.
I20260924 16:09:30.819394 27846 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2, e780221940044f909a4b7a5b9a6e293d; no voters:
I20260924 16:09:30.819594 27965 raft_consensus.cc:2781] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:30.819821 27965 raft_consensus.cc:686] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 LEADER]: Becoming Leader. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Running, Role: LEADER
I20260924 16:09:30.820278 27965 consensus_queue.cc:237] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:30.823177 27770 catalog_manager.cc:5495] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65). New cstate: current_term: 1 leader_uuid: "3e2e8173142f4fe6bef28bec93db39a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } health_report { overall_health: HEALTHY } } }
I20260924 16:09:30.856160 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.852141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.13.8.65:35387)
0924 16:09:30.852496 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:30.852503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:30.852516 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:09:30.852590 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:30.852592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:30.852647 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:30.852777 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:09:30.853037 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.853614 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.854599 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:30.854844 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:30.854959 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:30.855261 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:09:30.855727 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:09:30.855859 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:30.855922 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 16:09:30.855978 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260924 16:09:30.860421 28053 raft_consensus.cc:1261] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Refusing update from remote peer 3e2e8173142f4fe6bef28bec93db39a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:30.860426 28188 raft_consensus.cc:1261] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Refusing update from remote peer 3e2e8173142f4fe6bef28bec93db39a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:30.861244 28252 consensus_queue.cc:1035] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:30.861451 27965 consensus_queue.cc:1035] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:30.869328 28256 mvcc.cc:205] Tried to move back new op lower bound from 7332930235836649472 to 7332930235682086912. Current Snapshot: MvccSnapshot[applied={T|T < 7332930235836649472}]
I20260924 16:09:30.870031 28257 mvcc.cc:205] Tried to move back new op lower bound from 7332930235836649472 to 7332930235682086912. Current Snapshot: MvccSnapshot[applied={T|T < 7332930235836649472}]
I20260924 16:09:30.875994 27769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:30.876145 27769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:30.878672 27769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Sending DeleteTablet for 3 replicas of tablet 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:30.879367 28168 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d44eca229f14baebdf78f71ef947c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:30 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:41630
I20260924 16:09:30.879505 27893 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d44eca229f14baebdf78f71ef947c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:30 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:46834
I20260924 16:09:30.879597 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27808
I20260924 16:09:30.880460 28265 tablet_replica.cc:330] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: stopping tablet replica
I20260924 16:09:30.880694 28032 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d44eca229f14baebdf78f71ef947c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:30 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:47646
I20260924 16:09:30.880860 28265 raft_consensus.cc:2229] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:30.881037 28266 tablet_replica.cc:330] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: stopping tablet replica
I20260924 16:09:30.881198 28265 raft_consensus.cc:2258] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:30.881208 28266 raft_consensus.cc:2229] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:30.881349 28266 raft_consensus.cc:2258] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:30.882390 28266 ts_tablet_manager.cc:1905] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:30.882895 28265 ts_tablet_manager.cc:1905] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:30.885154 28265 ts_tablet_manager.cc:1918] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:30.885269 28265 log.cc:1196] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:30.885560 28265 ts_tablet_manager.cc:1939] T 1d44eca229f14baebdf78f71ef947c24 P 6123643ed5594297b8b0fa157e9ca807: Deleting consensus metadata
I20260924 16:09:30.886032 28266 ts_tablet_manager.cc:1918] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:30.886132 28266 log.cc:1196] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:30.886476 28266 ts_tablet_manager.cc:1939] T 1d44eca229f14baebdf78f71ef947c24 P e780221940044f909a4b7a5b9a6e293d: Deleting consensus metadata
I20260924 16:09:30.886925 27757 catalog_manager.cc:4841] TS 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): tablet 1d44eca229f14baebdf78f71ef947c24 (table test-table [id=66aa85a89ecf41fdb6866f18a83fbcc1]) successfully deleted
I20260924 16:09:30.887223 27756 catalog_manager.cc:4841] TS e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): tablet 1d44eca229f14baebdf78f71ef947c24 (table test-table [id=66aa85a89ecf41fdb6866f18a83fbcc1]) successfully deleted
W20260924 16:09:30.888724 27756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:35387 (error 108)
I20260924 16:09:30.888891 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27964
I20260924 16:09:30.889986 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:30.889567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:35387 (local address 127.0.0.1:46850)
0924 16:09:30.889792 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:30.889880 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:35387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260924 16:09:30.890170 27756 catalog_manager.cc:4584] TS 3e2e8173142f4fe6bef28bec93db39a2 (127.13.8.65:35387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d44eca229f14baebdf78f71ef947c24: Network error: Client connection negotiation failed: client connection to 127.13.8.65:35387: connect: Connection refused (error 111)
I20260924 16:09:30.895799 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28103
I20260924 16:09:30.901762 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 27737
I20260924 16:09:30.908835 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--webserver_interface=127.13.8.126
--webserver_port=33075
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:31.021332 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:31.021615 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:31.021656 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:31.025622 28268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:31.025677 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:31.025695 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:31.025730 28268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:31.025745 28268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:31.031087 28268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=33075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:31.032150 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:31.033187 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:31.041781 28276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.041772 28274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.041839 28273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:31.042104 28268 server_base.cc:1034] running on GCE node
I20260924 16:09:31.042721 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:31.043807 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:31.044967 28268 hybrid_clock.cc:648] HybridClock initialized: now 1790266171044952 us; error 24 us; skew 500 ppm
Sep 24 16:09:31 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:31.047791 28268 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:31.048708 28268 webserver.cc:460] Webserver started at http://127.13.8.126:33075/ using document root <none> and password file <none>
I20260924 16:09:31.049259 28268 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:31.049319 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:31.053056 28268 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260924 16:09:31.055222 28283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.056546 28268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:09:31.056692 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "b66e29737fcf4f4c86266665109560f3"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "df5b3c2e793884589bcd549ec7dbaa9b"
server_key_iv: "085e3c16539a9a31ecfed15e35246035"
server_key_version: "encryptionkey@0"
I20260924 16:09:31.057231 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:31.079095 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:31.082473 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:31.082682 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:31.090180 28268 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35247
I20260924 16:09:31.090193 28335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35247 every 8 connection(s)
I20260924 16:09:31.091338 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:31.095170 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28268
I20260924 16:09:31.095593 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:35387
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=40777
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:31.095643 28336 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:09:31.097254 28336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap starting.
I20260924 16:09:31.120520 28336 log.cc:824] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:31.132033 28336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:31.132418 28336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: Bootstrap complete.
I20260924 16:09:31.136736 28336 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:31.137199 28336 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Initialized, Role: FOLLOWER
I20260924 16:09:31.137746 28336 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:31.137897 28336 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:31.137995 28336 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:31.138116 28336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 2 FOLLOWER]: Advancing to term 3
I20260924 16:09:31.140100 28336 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:31.140436 28336 leader_election.cc:302] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66e29737fcf4f4c86266665109560f3; no voters:
I20260924 16:09:31.140779 28336 leader_election.cc:288] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 16:09:31.140875 28340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 16:09:31.141108 28340 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [term 3 LEADER]: Becoming Leader. State: Replica: b66e29737fcf4f4c86266665109560f3, State: Running, Role: LEADER
I20260924 16:09:31.141472 28340 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } }
I20260924 16:09:31.142045 28336 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:31.142966 28342 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66e29737fcf4f4c86266665109560f3. Latest consensus state: current_term: 3 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:31.143007 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b66e29737fcf4f4c86266665109560f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66e29737fcf4f4c86266665109560f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35247 } } }
I20260924 16:09:31.143096 28342 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:31.143128 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:31.147495 28355 catalog_manager.cc:1259] Loaded cluster ID: 7fb094e97dd94738b8bbf6ffbfe693f4
I20260924 16:09:31.147567 28355 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: loading cluster ID for follower catalog manager: success
W20260924 16:09:31.148594 28355 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b66e29737fcf4f4c86266665109560f3: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260924 16:09:31.148669 28355 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260924 16:09:31.148739 28349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:31.151451 28349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:31.151947 28349 catalog_manager.cc:1259] Loaded cluster ID: 7fb094e97dd94738b8bbf6ffbfe693f4
I20260924 16:09:31.152019 28349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 16:09:31.152618 28349 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f36f751b9ad google::LogMessage::Fail() at ??:0
@ 0x7f36f751e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f36f751b4de google::LogMessage::Flush() at ??:0
@ 0x7f36f751ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3701e5efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f3701e5fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3701e5c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3701ea0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f36f976d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f36f976dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f36f976f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f36f975e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f36fada26db start_thread at ??:0
@ 0x7f36f601961f clone at ??:0
W20260924 16:09:31.238327 28338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:31.238554 28338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:31.238628 28338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:31.242763 28338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:31.242815 28338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:31.242897 28338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:31.248121 28338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:35387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=40777
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:31.249188 28338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:31.250309 28338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:31.260002 28377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.260061 28378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.260126 28380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:31.260545 28338 server_base.cc:1034] running on GCE node
I20260924 16:09:31.260985 28338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:31.261662 28338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:31.262846 28338 hybrid_clock.cc:648] HybridClock initialized: now 1790266171262807 us; error 59 us; skew 500 ppm
Sep 24 16:09:31 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:31.265647 28338 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:31.266673 28338 webserver.cc:460] Webserver started at http://127.13.8.65:40777/ using document root <none> and password file <none>
I20260924 16:09:31.267254 28338 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:31.267344 28338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:31.271245 28338 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:09:31.273626 28387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.274959 28338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:31.275089 28338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "3e2e8173142f4fe6bef28bec93db39a2"
format_stamp: "Formatted at 2026-09-24 16:09:28 on dist-test-slave-vj6n"
server_key: "968ed30e529c0c714504fcab8e73c2de"
server_key_iv: "cc11950bed790c817232656f8eccc1a6"
server_key_version: "encryptionkey@0"
I20260924 16:09:31.275626 28338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:31.290395 28338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:31.293762 28338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:31.293967 28338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:31.294651 28338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:31.295789 28394 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 16:09:31.301627 28338 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 16:09:31.301718 28338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260924 16:09:31.301823 28338 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 16:09:31.303570 28338 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 16:09:31.303646 28338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:31.303944 28394 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap starting.
I20260924 16:09:31.314491 28338 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:35387
I20260924 16:09:31.314508 28501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:35387 every 8 connection(s)
I20260924 16:09:31.315583 28338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:31.323127 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28338
I20260924 16:09:31.323669 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:38687
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=35595
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:31.328256 28394 log.cc:824] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:31.335527 28394 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:09:31.335901 28394 tablet_bootstrap.cc:492] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Bootstrap complete.
I20260924 16:09:31.337054 28394 ts_tablet_manager.cc:1397] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.008s
I20260924 16:09:31.339600 28394 raft_consensus.cc:348] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:31.340030 28394 raft_consensus.cc:729] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2e8173142f4fe6bef28bec93db39a2, State: Initialized, Role: FOLLOWER
I20260924 16:09:31.340540 28394 consensus_queue.cc:260] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:31.341491 28394 ts_tablet_manager.cc:1428] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260924 16:09:31.382620 28504 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:09:31.317732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:60857)
0924 16:09:31.318259 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.318315 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:31.319158 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:31.381785 (+ 62627us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":143,"threads_started":1}
W20260924 16:09:31.383162 28502 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:35247 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:35247: Client connection negotiation failed: client connection to 127.13.8.126:35247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 16:09:31.383941 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.383693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:57445)
0924 16:09:31.383823 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.383885 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:09:31.384577 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.384370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:55779)
0924 16:09:31.384474 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.384528 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260924 16:09:31.384742 28502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:09:31.455031 28505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:31.455382 28505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:31.455497 28505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:31.460009 28505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:31.460137 28505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:31.460251 28505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:31.465844 28505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:38687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=35595
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:31.467121 28505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:31.468287 28505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:31.478963 28515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.478973 28512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.479029 28513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:31.479228 28505 server_base.cc:1034] running on GCE node
I20260924 16:09:31.479715 28505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:31.480434 28505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:31.481616 28505 hybrid_clock.cc:648] HybridClock initialized: now 1790266171481595 us; error 29 us; skew 500 ppm
Sep 24 16:09:31 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:31.484470 28505 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:31.485339 28505 webserver.cc:460] Webserver started at http://127.13.8.66:35595/ using document root <none> and password file <none>
I20260924 16:09:31.485894 28505 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:31.485968 28505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:31.489933 28505 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:09:31.492225 28522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.493505 28505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:09:31.493613 28505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "e780221940044f909a4b7a5b9a6e293d"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "0addfe383f55940258e5c3864409b7a4"
server_key_iv: "9031424ac3795d68b866dbf5d145822f"
server_key_version: "encryptionkey@0"
I20260924 16:09:31.494175 28505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:31.512820 28505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:31.515971 28505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:31.516182 28505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:31.516790 28505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:31.517603 28505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:31.517649 28505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.517690 28505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:31.517710 28505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.526733 28505 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:38687
I20260924 16:09:31.526757 28635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:38687 every 8 connection(s)
I20260924 16:09:31.527781 28505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:31.531522 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28505
I20260924 16:09:31.531996 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:43565
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:35247
--webserver_port=41517
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:09:31.532023 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.530006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:58727)
0924 16:09:31.530548 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.530987 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":174,"threads_started":1}
W20260924 16:09:31.532471 28636 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:35247 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:35247: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
I20260924 16:09:31.533217 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.532917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:50289)
0924 16:09:31.533063 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.533130 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 16:09:31.533950 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.533747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:46637)
0924 16:09:31.533850 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.533890 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 16:09:31.534122 28636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:09:31.645072 28639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:31.645377 28639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:31.645457 28639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:31.649570 28639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:31.649669 28639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:31.649763 28639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:31.655051 28639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:43565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=41517
--tserver_master_addrs=127.13.8.126:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:31.656154 28639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:31.657243 28639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:31.666997 28646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:31.667028 28644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:31.667137 28639 server_base.cc:1034] running on GCE node
W20260924 16:09:31.667028 28648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:31.667611 28639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:31.668220 28639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:31.669376 28639 hybrid_clock.cc:648] HybridClock initialized: now 1790266171669362 us; error 25 us; skew 500 ppm
Sep 24 16:09:31 dist-test-slave-vj6n krb5kdc[27183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:31.672106 28639 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:31.673004 28639 webserver.cc:460] Webserver started at http://127.13.8.67:41517/ using document root <none> and password file <none>
I20260924 16:09:31.673579 28639 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:31.673674 28639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:31.677522 28639 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:09:31.679731 28655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.681015 28639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:31.681165 28639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "6123643ed5594297b8b0fa157e9ca807"
format_stamp: "Formatted at 2026-09-24 16:09:29 on dist-test-slave-vj6n"
server_key: "8a44395f4bd67ab2df99df96181ccab6"
server_key_iv: "22311eb53d2c45ae7487b7f97c59208d"
server_key_version: "encryptionkey@0"
I20260924 16:09:31.681704 28639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:31.714743 28639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:31.718047 28639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:31.718325 28639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:31.718982 28639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:31.719882 28639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:31.719954 28639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.720028 28639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:31.720072 28639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:31.729593 28639 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:43565
I20260924 16:09:31.729627 28768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:43565 every 8 connection(s)
I20260924 16:09:31.730676 28639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:09:31.734556 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.732829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:56483)
0924 16:09:31.733309 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.733761 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":103,"threads_started":1}
W20260924 16:09:31.735111 28769 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:35247 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:35247: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
I20260924 16:09:31.735797 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.735533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:58901)
0924 16:09:31.735666 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.735740 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 16:09:31.736434 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:31.736236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:54359)
0924 16:09:31.736342 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:31.736385 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 16:09:31.736603 28769 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 16:09:31.739183 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28639
I20260924 16:09:32.385902 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.385362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:54345)
0924 16:09:32.385692 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:32.385776 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
I20260924 16:09:32.512895 28774 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:32.513164 28774 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:32.514853 28774 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
I20260924 16:09:32.535120 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.534639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:52483)
0924 16:09:32.534920 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:32.534980 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
Sep 24 16:09:32 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.66@KRBTEST.COM
Sep 24 16:09:32 dist-test-slave-vj6n krb5kdc[27183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266171, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.67@KRBTEST.COM
I20260924 16:09:32.541580 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.515377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:56129 (local address 127.13.8.66:38687)
0924 16:09:32.515891 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:32.515905 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:32.515945 (+ 40us) server_negotiation.cc:407] Connection header received
0924 16:09:32.516739 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:32.516750 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:32.517337 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:32.517783 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:32.519219 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.520775 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.522047 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.522889 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.523022 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:32.536692 (+ 13670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:32.536701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:32.536722 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:32.536759 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:32.538917 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539593 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:32.539603 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:32.539608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:32.539719 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:32.540099 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:32.540101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:32.540103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:32.540500 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:32.540643 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:32.541174 (+ 531us) server_negotiation.cc:299] Negotiation successful
0924 16:09:32.541424 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":102,"threads_started":1}
I20260924 16:09:32.541581 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.515098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:43565 (local address 127.13.8.65:37233)
0924 16:09:32.515466 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:32.515477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:32.515604 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:32.519948 (+ 4344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:32.519951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:32.520110 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:32.520209 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:32.520215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.521643 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.521647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:32.522519 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:32.522531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.524187 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.524191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:32.524280 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:32.534550 (+ 10270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:32.534562 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:32.536779 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:32.539152 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:32.539156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:32.539447 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:32.539805 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:32.539812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:32.539963 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:32.540619 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:32.540628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:32.540867 (+ 239us) client_negotiation.cc:769] Sending connection context
0924 16:09:32.541055 (+ 188us) client_negotiation.cc:240] Negotiation successful
0924 16:09:32.541396 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":72,"spinlock_wait_cycles":17792,"thread_start_us":76,"threads_started":1}
I20260924 16:09:32.541646 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.515403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:37233 (local address 127.13.8.67:43565)
0924 16:09:32.518459 (+ 3056us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:32.518471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:32.518504 (+ 33us) server_negotiation.cc:407] Connection header received
0924 16:09:32.519233 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:32.519240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:32.519679 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:32.519987 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:32.520393 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.521511 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.523359 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.524050 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.524231 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:32.537014 (+ 12783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:32.537019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:32.537031 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:32.537063 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:32.539031 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539567 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:32.539574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:32.539579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:32.539703 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:32.540064 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:32.540068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:32.540073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:32.540503 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:32.540643 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:32.541166 (+ 523us) server_negotiation.cc:299] Negotiation successful
0924 16:09:32.541453 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2931,"thread_start_us":82,"threads_started":1}
I20260924 16:09:32.541989 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.515084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:38687 (local address 127.13.8.65:56129)
0924 16:09:32.515459 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:32.515476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:32.515584 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:32.517918 (+ 2334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:32.517925 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:32.518467 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:32.519005 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:32.519023 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.520923 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.520926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:32.521897 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:32.521908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:32.523012 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:32.523019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:32.523261 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:32.534530 (+ 11269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:32.534558 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:32.536427 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:32.539062 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539072 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:32.539089 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:32.539446 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:32.539846 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:32.539849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:32.539850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:32.539970 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:32.540619 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:32.540625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:32.540867 (+ 242us) client_negotiation.cc:769] Sending connection context
0924 16:09:32.541055 (+ 188us) client_negotiation.cc:240] Negotiation successful
0924 16:09:32.541495 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":165,"spinlock_wait_cycles":18176}
W20260924 16:09:32.544858 28391 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
W20260924 16:09:32.545037 28390 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:32.545529 28390 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2; no voters: 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d
I20260924 16:09:32.545840 28774 raft_consensus.cc:2726] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:09:32.737468 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:32.737090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:54841)
0924 16:09:32.737322 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:32.737372 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260924 16:09:33.387107 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:33.386689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:37871)
0924 16:09:33.386949 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:33.387013 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260924 16:09:33.536319 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:33.535888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:35929)
0924 16:09:33.536168 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:33.536227 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
I20260924 16:09:33.738583 28782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:33.738153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:33223)
0924 16:09:33.738428 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:33.738477 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260924 16:09:34.388247 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:34.387806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:54961)
0924 16:09:34.388097 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:34.388151 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260924 16:09:34.505681 28784 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:34.505841 28784 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:34.506330 28784 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
W20260924 16:09:34.507191 28391 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
W20260924 16:09:34.507279 28390 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:34.507395 28390 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2; no voters: 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d
I20260924 16:09:34.507596 28784 raft_consensus.cc:2726] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:09:34.537714 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:34.537237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:53465)
0924 16:09:34.537529 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:34.537592 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260924 16:09:34.739778 28786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:34.739329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:42057)
0924 16:09:34.739629 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:34.739684 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260924 16:09:35.389426 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:35.389017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:58669)
0924 16:09:35.389276 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:35.389330 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260924 16:09:35.538887 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:35.538439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:48941)
0924 16:09:35.538701 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:35.538757 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260924 16:09:35.741022 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:35.740618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:58375)
0924 16:09:35.740870 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:35.740929 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260924 16:09:36.375490 28790 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:36.375702 28790 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:36.376255 28790 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
W20260924 16:09:36.377100 28391 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
W20260924 16:09:36.377190 28390 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:36.377285 28390 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2; no voters: 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d
I20260924 16:09:36.377490 28790 raft_consensus.cc:2726] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:09:36.390880 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:36.390419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:53445)
0924 16:09:36.390733 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:36.390787 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":104,"threads_started":1}
I20260924 16:09:36.540050 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:36.539658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:39201)
0924 16:09:36.539908 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:36.539960 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260924 16:09:36.742096 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:36.741704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:33763)
0924 16:09:36.741950 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:36.742002 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260924 16:09:37.392134 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:37.391715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:39187)
0924 16:09:37.391972 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:37.392032 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260924 16:09:37.541216 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:37.540802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:56101)
0924 16:09:37.541054 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:37.541117 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260924 16:09:37.743176 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:37.742786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:49381)
0924 16:09:37.743031 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:37.743083 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260924 16:09:38.393301 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:38.392895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:37779)
0924 16:09:38.393144 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:38.393197 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260924 16:09:38.542694 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:38.542001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:42577)
0924 16:09:38.542532 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:38.542594 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":86,"threads_started":1}
I20260924 16:09:38.744315 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:38.743852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:40703)
0924 16:09:38.744172 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:38.744223 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":94,"threads_started":1}
I20260924 16:09:39.017975 28800 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:39.018113 28800 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:39.018637 28800 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
W20260924 16:09:39.019520 28390 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
W20260924 16:09:39.019663 28391 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:39.019832 28391 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2; no voters: 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d
I20260924 16:09:39.020052 28800 raft_consensus.cc:2726] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:09:39.394500 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:39.394049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:35025)
0924 16:09:39.394346 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:39.394402 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":75,"threads_started":1}
I20260924 16:09:39.543836 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:39.543440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:56511)
0924 16:09:39.543696 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:39.543746 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260924 16:09:39.745550 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:39.745127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:53475)
0924 16:09:39.745390 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:39.745447 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
I20260924 16:09:40.395678 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:40.395247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:47101)
0924 16:09:40.395518 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:40.395577 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260924 16:09:40.545046 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:40.544624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:38999)
0924 16:09:40.544894 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:40.544949 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260924 16:09:40.746811 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:40.746321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:35143)
0924 16:09:40.746666 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:40.746721 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":128,"threads_started":1}
I20260924 16:09:41.397167 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:41.396686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:33559)
0924 16:09:41.396992 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:41.397065 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
I20260924 16:09:41.546343 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:41.545855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:36393)
0924 16:09:41.546131 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:41.546187 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":98,"threads_started":1}
I20260924 16:09:41.748065 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:41.747668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:42503)
0924 16:09:41.747919 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:41.747970 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
I20260924 16:09:42.398481 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:42.398020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:44745)
0924 16:09:42.398326 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:42.398383 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260924 16:09:42.547617 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:42.547190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:42957)
0924 16:09:42.547455 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:42.547514 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260924 16:09:42.749132 28812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:42.748742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:44103)
0924 16:09:42.748985 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:42.749038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260924 16:09:43.399641 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:43.399218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:50999)
0924 16:09:43.399482 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:43.399542 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260924 16:09:43.548933 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:43.548499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:43239)
0924 16:09:43.548770 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:43.548831 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260924 16:09:43.750473 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:43.749982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:50427)
0924 16:09:43.750292 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:43.750352 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260924 16:09:44.401010 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:44.400530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:47609)
0924 16:09:44.400834 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:44.400905 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
I20260924 16:09:44.550403 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:44.549860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:49799)
0924 16:09:44.550158 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:44.550286 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260924 16:09:44.751827 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:44.751350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:41881)
0924 16:09:44.751664 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:44.751724 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":88,"threads_started":1}
I20260924 16:09:44.785265 28819 raft_consensus.cc:482] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:44.785394 28819 raft_consensus.cc:504] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6123643ed5594297b8b0fa157e9ca807" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 43565 } } peers { permanent_uuid: "e780221940044f909a4b7a5b9a6e293d" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 38687 } } peers { permanent_uuid: "3e2e8173142f4fe6bef28bec93db39a2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 35387 } }
I20260924 16:09:44.785928 28819 leader_election.cc:288] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565), e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687)
W20260924 16:09:44.786777 28390 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e780221940044f909a4b7a5b9a6e293d (127.13.8.66:38687): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
W20260924 16:09:44.786906 28391 leader_election.cc:341] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6123643ed5594297b8b0fa157e9ca807 (127.13.8.67:43565): Not found: Tablet not found: 1d44eca229f14baebdf78f71ef947c24
I20260924 16:09:44.787034 28391 leader_election.cc:302] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e2e8173142f4fe6bef28bec93db39a2; no voters: 6123643ed5594297b8b0fa157e9ca807, e780221940044f909a4b7a5b9a6e293d
I20260924 16:09:44.787214 28819 raft_consensus.cc:2726] T 1d44eca229f14baebdf78f71ef947c24 P 3e2e8173142f4fe6bef28bec93db39a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:09:45.402486 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:45.401921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:48429)
0924 16:09:45.402325 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:45.402382 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":92,"threads_started":1}
I20260924 16:09:45.551820 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:45.551282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:38127)
0924 16:09:45.551645 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:45.551719 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":140,"threads_started":1}
I20260924 16:09:45.753072 28822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:45.752620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:55601)
0924 16:09:45.752903 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:45.752973 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":89,"threads_started":1}
I20260924 16:09:46.403972 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:46.403481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.65:53385)
0924 16:09:46.403806 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:46.403875 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":85,"threads_started":1}
I20260924 16:09:46.553224 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:46.552757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.66:54145)
0924 16:09:46.553064 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:46.553125 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
I20260924 16:09:46.749814 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28338
I20260924 16:09:46.754520 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:46.754052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35247 (local address 127.13.8.67:45467)
0924 16:09:46.754372 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:46.754426 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:35247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":87,"threads_started":1}
I20260924 16:09:46.756637 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28505
I20260924 16:09:46.762830 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28639
2026-09-24T16:09:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19456 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:09:46 dist-test-slave-vj6n krb5kdc[28828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:09:46 dist-test-slave-vj6n krb5kdc[28828](info): set up 2 sockets
Sep 24 16:09:46 dist-test-slave-vj6n krb5kdc[28828](info): commencing operation
krb5kdc: starting...
W20260924 16:09:47.807279 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:09:47 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:09:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:09:47.929849 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:48.042038 28841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:48.042366 28841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:48.042444 28841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:48.046738 28841 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 16:09:48.046816 28841 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 16:09:48.046844 28841 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 16:09:48.046875 28841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:09:48.046918 28841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:48.046957 28841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:48.046993 28841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:09:48.047029 28841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:09:48.052290 28841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:48.053401 28841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:48.054498 28841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:48.063098 28849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:48.063162 28846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.063218 28841 server_base.cc:1034] running on GCE node
W20260924 16:09:48.063411 28847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.064005 28841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:48.065169 28841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:48.066357 28841 hybrid_clock.cc:648] HybridClock initialized: now 1790266188066321 us; error 45 us; skew 500 ppm
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:48.076834 28841 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:09:48.077845 28841 webserver.cc:460] Webserver started at http://127.13.8.126:42331/ using document root <none> and password file <none>
I20260924 16:09:48.078537 28841 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:48.078631 28841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:48.078887 28841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:48.080981 28841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "29e97d652c4d4503b923ba6c507e2149"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "e3ed4625065fc6c85c8430bbb9131b3c"
server_key_iv: "b3f43bc312387e2435f5853263ea675a"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.081552 28841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "29e97d652c4d4503b923ba6c507e2149"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "e3ed4625065fc6c85c8430bbb9131b3c"
server_key_iv: "b3f43bc312387e2435f5853263ea675a"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.085526 28841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:09:48.088043 28856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.089311 28841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:09:48.089459 28841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "29e97d652c4d4503b923ba6c507e2149"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "e3ed4625065fc6c85c8430bbb9131b3c"
server_key_iv: "b3f43bc312387e2435f5853263ea675a"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.089573 28841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:48.107187 28841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:48.110692 28841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:48.110922 28841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:48.118784 28841 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41443
I20260924 16:09:48.119093 28908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41443 every 8 connection(s)
I20260924 16:09:48.119866 28841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:09:48.122855 28909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:48.125943 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28841
I20260924 16:09:48.126089 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:09:48.126461 13345 external_mini_cluster.cc:1428] Setting key c9c76c0f2c75ece276ae1a9193393116
I20260924 16:09:48.129315 28909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap starting.
I20260924 16:09:48.131824 28909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:48.132640 28909 log.cc:824] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:48.134809 28909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: No bootstrap required, opened a new log
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:48.138464 28909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:09:48.138710 28909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:48.138782 28909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Initialized, Role: FOLLOWER
I20260924 16:09:48.139258 28909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:09:48.139384 28909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:09:48.139470 28909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:09:48.139587 28909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:48.140535 28909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:09:48.140898 28909 leader_election.cc:302] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e97d652c4d4503b923ba6c507e2149; no voters:
I20260924 16:09:48.141281 28909 leader_election.cc:288] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:09:48.141372 28914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:48.141649 28914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 LEADER]: Becoming Leader. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Running, Role: LEADER
I20260924 16:09:48.142031 28914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:09:48.142522 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.127992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51690 (local address 127.13.8.126:41443)
0924 16:09:48.128517 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.128534 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.128574 (+ 40us) server_negotiation.cc:407] Connection header received
0924 16:09:48.129265 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.129284 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.129725 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.130106 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:48.131035 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.132142 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.133099 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.133597 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.133775 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.136557 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:48.136562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:48.136576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:48.136603 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:48.139105 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.139641 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.139648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.139654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.139829 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.140124 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.140128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.140129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.140565 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:48.140779 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.141066 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.141431 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":172,"threads_started":1}
I20260924 16:09:48.142922 28909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:09:48.143486 28914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:09:48.143947 28915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e97d652c4d4503b923ba6c507e2149. Latest consensus state: current_term: 1 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:09:48.144793 28915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:48.145067 28914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:09:48.145393 28926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:09:48.148941 28926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:09:48.155066 28926 catalog_manager.cc:1347] Generated new cluster ID: e3ec742ea41d40c09e6386972eedb34c
I20260924 16:09:48.155146 28926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:09:48.160626 28926 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:09:48.161846 28926 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:09:48.167229 28926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Generated new TSK 0
I20260924 16:09:48.167994 28926 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:09:48.201349 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:48.321424 28935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:48.321691 28935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:48.321751 28935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:48.325978 28935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:48.326045 28935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:48.326135 28935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:09:48.331377 28935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:48.332455 28935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:48.333518 28935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:48.343982 28941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:48.344027 28940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.344139 28935 server_base.cc:1034] running on GCE node
W20260924 16:09:48.344261 28943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.344875 28935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:48.345527 28935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:48.346719 28935 hybrid_clock.cc:648] HybridClock initialized: now 1790266188346701 us; error 37 us; skew 500 ppm
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:48.349452 28935 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:09:48.350314 28935 webserver.cc:460] Webserver started at http://127.13.8.65:44815/ using document root <none> and password file <none>
I20260924 16:09:48.350850 28935 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:48.350906 28935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:48.351081 28935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:48.353036 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "ab74bc2dea444552a634f85875452681"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "a02470538d42df98c0a79de8f8df80cf"
server_key_iv: "5a6e3e26c494973791d763140fd325ce"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.353534 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "ab74bc2dea444552a634f85875452681"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "a02470538d42df98c0a79de8f8df80cf"
server_key_iv: "5a6e3e26c494973791d763140fd325ce"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.357088 28935 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260924 16:09:48.359412 28950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.360592 28935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:09:48.360682 28935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "ab74bc2dea444552a634f85875452681"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "a02470538d42df98c0a79de8f8df80cf"
server_key_iv: "5a6e3e26c494973791d763140fd325ce"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.360759 28935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:48.370536 28935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:48.373660 28935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:48.373803 28935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:48.374373 28935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:48.375159 28935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:48.375204 28935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.375242 28935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:48.375257 28935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.383909 28935 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:44961
I20260924 16:09:48.383978 29063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:44961 every 8 connection(s)
I20260924 16:09:48.384781 28935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:09:48.387703 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 28935
I20260924 16:09:48.387842 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:09:48.388170 13345 external_mini_cluster.cc:1428] Setting key 8a0e5a79a768f5b2ea8db7c2d2f5aae5
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:09:48.400982 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.386700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:33029 (local address 127.13.8.126:41443)
0924 16:09:48.386855 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.386860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.387856 (+ 996us) server_negotiation.cc:407] Connection header received
0924 16:09:48.389541 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.389544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.389590 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.389711 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:48.391476 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.392359 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.394337 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.394615 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.394699 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.397529 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:48.397532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:48.397534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:48.397556 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:48.399130 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.399628 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.399630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.399632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.399706 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.400007 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.400009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.400010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.400176 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:48.400254 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.400764 (+ 510us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.400863 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:09:48.401834 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.387125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:33029)
0924 16:09:48.387678 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.387740 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.389303 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.389974 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.389983 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.390582 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:48.391306 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.391328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.392519 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.392523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.394145 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.394160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.394782 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.394787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.395025 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.395652 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:48.395679 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:48.397359 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:48.399237 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.399242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.399256 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.399526 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.399790 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.399792 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.399794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.399925 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.400268 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:48.400273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:48.400500 (+ 227us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.400703 (+ 203us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.400953 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":114,"threads_started":1}
I20260924 16:09:48.403391 29064 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:09:48.403648 29064 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:48.404142 29064 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:09:48.405646 28873 ts_manager.cc:194] Registered new tserver with Master: ab74bc2dea444552a634f85875452681 (127.13.8.65:44961)
I20260924 16:09:48.406826 28873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:33029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:09:48.417454 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:48.528430 29069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:48.528661 29069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:48.528699 29069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:48.532517 29069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:48.532562 29069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:48.532613 29069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:09:48.537602 29069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:48.538607 29069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:48.539528 29069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:48.549105 29075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:48.549099 29074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.549171 29069 server_base.cc:1034] running on GCE node
W20260924 16:09:48.549099 29077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.549685 29069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:48.550370 29069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:48.551548 29069 hybrid_clock.cc:648] HybridClock initialized: now 1790266188551517 us; error 42 us; skew 500 ppm
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:48.554177 29069 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:09:48.555025 29069 webserver.cc:460] Webserver started at http://127.13.8.66:45633/ using document root <none> and password file <none>
I20260924 16:09:48.555580 29069 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:48.555670 29069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:48.555895 29069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:48.557821 29069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "86ac61ebeca58ff2a011600f8cdb6e48"
server_key_iv: "a0ccfae9448bfb1bedd6c21ca0a123d6"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.558420 29069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "86ac61ebeca58ff2a011600f8cdb6e48"
server_key_iv: "a0ccfae9448bfb1bedd6c21ca0a123d6"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.562399 29069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:09:48.564846 29084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.566041 29069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:09:48.566192 29069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "86ac61ebeca58ff2a011600f8cdb6e48"
server_key_iv: "a0ccfae9448bfb1bedd6c21ca0a123d6"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.566359 29069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:48.579557 29069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:48.582401 29069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:48.582607 29069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:48.583192 29069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:48.584012 29069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:48.584082 29069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.584156 29069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:48.584206 29069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.593255 29069 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:40871
I20260924 16:09:48.593307 29197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:40871 every 8 connection(s)
I20260924 16:09:48.594208 29069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:09:48.603505 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29069
I20260924 16:09:48.603645 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:09:48.603935 13345 external_mini_cluster.cc:1428] Setting key ac864bc1c68fa5d88a3b4a25a6f14462
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:48.609270 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.596065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:43485 (local address 127.13.8.126:41443)
0924 16:09:48.596227 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.596231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.596994 (+ 763us) server_negotiation.cc:407] Connection header received
0924 16:09:48.598403 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.598407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.598471 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.598591 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:48.599988 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.600653 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.602372 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.602834 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.602982 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.605704 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:48.605708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:48.605711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:48.605739 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:48.607220 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.607743 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.607748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.607750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.607837 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.608191 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.608195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.608197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.608386 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:48.608497 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.608986 (+ 489us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.609110 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 16:09:48.610136 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.596395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:43485)
0924 16:09:48.596848 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.596891 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.598118 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.598771 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.598778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.599157 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:48.599836 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.599857 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.600788 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.600792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.602202 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.602237 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.603066 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.603072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.603295 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.603932 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:48.603948 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:48.605540 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:48.607344 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.607350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.607366 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.607634 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.607933 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.607937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.607939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.608092 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.608495 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:48.608502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:48.608750 (+ 248us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.608942 (+ 192us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.609270 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
I20260924 16:09:48.611397 29198 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:09:48.611656 29198 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:48.612177 29198 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:09:48.613296 28873 ts_manager.cc:194] Registered new tserver with Master: 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
I20260924 16:09:48.614261 28873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:43485
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:09:48.633903 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:09:48.746842 29203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:09:48.747082 29203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:09:48.747123 29203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:09:48.751178 29203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:09:48.751233 29203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:09:48.751291 29203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:09:48.756493 29203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:09:48.757459 29203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:09:48.758508 29203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:09:48.768360 29208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:48.768356 29211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:09:48.768471 29209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:09:48.768592 29203 server_base.cc:1034] running on GCE node
I20260924 16:09:48.769068 29203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:09:48.769814 29203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:09:48.770977 29203 hybrid_clock.cc:648] HybridClock initialized: now 1790266188770939 us; error 48 us; skew 500 ppm
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:09:48.773770 29203 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:09:48.774681 29203 webserver.cc:460] Webserver started at http://127.13.8.67:45703/ using document root <none> and password file <none>
I20260924 16:09:48.775213 29203 fs_manager.cc:359] Metadata directory not provided
I20260924 16:09:48.775270 29203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:09:48.775439 29203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:09:48.777411 29203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "24d97933791052949e9ef12454d75d9d"
server_key_iv: "2dc9f3daf681d02288215bce53c94d81"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.777923 29203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "24d97933791052949e9ef12454d75d9d"
server_key_iv: "2dc9f3daf681d02288215bce53c94d81"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.781665 29203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:09:48.784091 29218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.785538 29203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:09:48.785668 29203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "24d97933791052949e9ef12454d75d9d"
server_key_iv: "2dc9f3daf681d02288215bce53c94d81"
server_key_version: "encryptionkey@0"
I20260924 16:09:48.785754 29203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:09:48.799120 29203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:09:48.802009 29203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:09:48.802155 29203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:09:48.802793 29203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:09:48.803581 29203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:09:48.803627 29203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.803665 29203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:09:48.803680 29203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:09:48.812026 29203 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:33069
I20260924 16:09:48.812047 29331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:33069 every 8 connection(s)
I20260924 16:09:48.812945 29203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:09:48.820151 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29203
I20260924 16:09:48.820274 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:09:48.820559 13345 external_mini_cluster.cc:1428] Setting key 0ef35319533a78beb4b4db0e7efd77b7
Sep 24 16:09:48 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:09:48.826730 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.814772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:54771 (local address 127.13.8.126:41443)
0924 16:09:48.814908 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.814912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.815695 (+ 783us) server_negotiation.cc:407] Connection header received
0924 16:09:48.816935 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.816939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.816988 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.817060 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:48.818154 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.818745 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.819817 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.820038 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.820118 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.823116 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:48.823119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:48.823120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:48.823143 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:48.824619 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.825173 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.825177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.825179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.825262 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.825616 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.825621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.825623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.825796 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:48.825904 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.826446 (+ 542us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.826567 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:09:48.827493 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.815079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:54771)
0924 16:09:48.815509 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.815551 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.816732 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.817225 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.817231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.817559 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:09:48.818026 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.818038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.818855 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.818858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.819690 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.819698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.820207 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.820215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.820419 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.821203 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:09:48.821223 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:09:48.822950 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:09:48.824751 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.824757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.824770 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.825069 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.825363 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:09:48.825366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:09:48.825368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:09:48.825521 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:09:48.825905 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:09:48.825912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:09:48.826151 (+ 239us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.826400 (+ 249us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.826714 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
I20260924 16:09:48.828500 29332 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:09:48.828740 29332 heartbeater.cc:464] Registering TS with master...
I20260924 16:09:48.829205 29332 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:09:48.830196 28873 ts_manager.cc:194] Registered new tserver with Master: bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069)
I20260924 16:09:48.830804 28873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:54771
I20260924 16:09:48.834410 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:09:48.843212 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.836329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.13.8.126:41443)
0924 16:09:48.836430 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.836433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.836671 (+ 238us) server_negotiation.cc:407] Connection header received
0924 16:09:48.836752 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.836755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.836795 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.836863 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:09:48.837544 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.838036 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.838872 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.839085 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.839165 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.840315 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:09:48.840318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:09:48.840319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:09:48.840340 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:09:48.841756 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.842105 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.842108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.842109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.842176 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:09:48.842458 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:09:48.842491 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:09:48.842494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:09:48.842721 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:09:48.842821 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.843020 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.843105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:09:48.846531 28873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:09:48.849516 28873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:09:48.864742 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.859112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.13.8.65:44961)
0924 16:09:48.859679 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.859695 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.859716 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:09:48.859789 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.859794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.859980 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.860465 (+ 485us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:48.860623 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.861664 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.863258 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.864020 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.864144 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.864300 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.864476 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.864584 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":113,"threads_started":1}
I20260924 16:09:48.864742 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.858294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33069 (local address 127.0.0.1:60820)
0924 16:09:48.858961 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.859008 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.859165 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.859979 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.859986 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.860011 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:48.860158 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.860166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.861647 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.861650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.863126 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.863138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.864280 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.864284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.864383 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.864408 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.864456 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.864588 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":66,"thread_start_us":80,"threads_started":1}
I20260924 16:09:48.864746 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.858407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.0.0.1:38650)
0924 16:09:48.859284 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.859298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.859387 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.860188 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.860194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.860216 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:48.860319 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.860326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.861786 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.861789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.863128 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.863138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.864145 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.864148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.864259 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.864300 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.864445 (+ 145us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.864621 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":81,"thread_start_us":113,"threads_started":1}
I20260924 16:09:48.865566 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.858440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.13.8.67:33069)
0924 16:09:48.858825 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.858837 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.859313 (+ 476us) server_negotiation.cc:407] Connection header received
0924 16:09:48.859401 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.859407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.859632 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.859787 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:48.860250 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.861523 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.863294 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.864113 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.865058 (+ 945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.865215 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.865266 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.865422 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":139,"threads_started":1}
I20260924 16:09:48.866740 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.859016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:40871 (local address 127.0.0.1:48120)
0924 16:09:48.860053 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.860065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.860168 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.862080 (+ 1912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.862083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.862102 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:48.862408 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.862419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.863776 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.863780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.865214 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.865226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.866325 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.866328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.866421 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.866443 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.866529 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.866617 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260924 16:09:48.866791 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.859436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.13.8.66:40871)
0924 16:09:48.859882 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.859893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.860165 (+ 272us) server_negotiation.cc:407] Connection header received
0924 16:09:48.861735 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.861742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.861920 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.862098 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:48.862543 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.863607 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.865350 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.866183 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.866327 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.866462 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.866557 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.866644 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":152,"threads_started":1}
I20260924 16:09:48.868027 28998 tablet_service.cc:1467] Processing CreateTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f (DEFAULT_TABLE table=test-table [id=8335941660684882a43ad2a6cbf146a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:48.868227 29266 tablet_service.cc:1467] Processing CreateTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f (DEFAULT_TABLE table=test-table [id=8335941660684882a43ad2a6cbf146a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:48.868963 28998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2847f91f8a0c40d5b508daa44b522e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:48.869192 29266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2847f91f8a0c40d5b508daa44b522e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:48.875363 29349 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Bootstrap starting.
I20260924 16:09:48.875634 29350 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Bootstrap starting.
I20260924 16:09:48.877709 29349 tablet_bootstrap.cc:654] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:48.877804 29350 tablet_bootstrap.cc:654] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:48.877532 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f (DEFAULT_TABLE table=test-table [id=8335941660684882a43ad2a6cbf146a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:09:48.878759 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2847f91f8a0c40d5b508daa44b522e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:09:48.878983 29349 log.cc:824] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:48.879138 29350 log.cc:824] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:48.881126 29349 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: No bootstrap required, opened a new log
I20260924 16:09:48.881327 29349 ts_tablet_manager.cc:1397] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:09:48.881399 29350 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: No bootstrap required, opened a new log
I20260924 16:09:48.881593 29350 ts_tablet_manager.cc:1397] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:09:48.884351 29350 raft_consensus.cc:348] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.884645 29350 raft_consensus.cc:374] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:48.884721 29350 raft_consensus.cc:729] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab74bc2dea444552a634f85875452681, State: Initialized, Role: FOLLOWER
I20260924 16:09:48.885154 29350 consensus_queue.cc:260] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.885296 29349 raft_consensus.cc:348] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.885610 29349 raft_consensus.cc:374] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:48.885872 29064 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:09:48.886135 29354 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Bootstrap starting.
I20260924 16:09:48.886382 29350 ts_tablet_manager.cc:1428] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 16:09:48.885702 29349 raft_consensus.cc:729] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda6532b1ecc4ebd80e248fd91a15d3a, State: Initialized, Role: FOLLOWER
I20260924 16:09:48.887554 29349 consensus_queue.cc:260] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.888212 29332 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:09:48.888464 29349 ts_tablet_manager.cc:1428] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260924 16:09:48.891273 29354 tablet_bootstrap.cc:654] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Neither blocks nor log segments found. Creating new log.
I20260924 16:09:48.891991 29354 log.cc:824] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Log is configured to *not* fsync() on all Append() calls
I20260924 16:09:48.893790 29354 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: No bootstrap required, opened a new log
I20260924 16:09:48.893994 29354 ts_tablet_manager.cc:1397] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260924 16:09:48.897001 29354 raft_consensus.cc:348] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.897217 29354 raft_consensus.cc:374] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:09:48.897286 29354 raft_consensus.cc:729] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 192a4c57b3c14a60ae5be9a8ce9184de, State: Initialized, Role: FOLLOWER
I20260924 16:09:48.897662 29354 consensus_queue.cc:260] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.898278 29198 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:09:48.898495 29354 ts_tablet_manager.cc:1428] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260924 16:09:48.918027 29355 raft_consensus.cc:482] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:09:48.918300 29355 raft_consensus.cc:504] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.919548 29355 leader_election.cc:288] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab74bc2dea444552a634f85875452681 (127.13.8.65:44961), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
I20260924 16:09:48.923558 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.919798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:40871 (local address 127.13.8.67:56017)
0924 16:09:48.919909 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.919948 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.920041 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.920330 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.920332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.920349 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:48.920495 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.920502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.921415 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.921418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.922384 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.922394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.923085 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.923088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.923224 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.923246 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.923319 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.923390 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 16:09:48.923623 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.919895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:56017 (local address 127.13.8.66:40871)
0924 16:09:48.920010 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.920015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.920031 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:09:48.920163 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.920166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.920223 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.920309 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:48.920637 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.921286 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.922512 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.922962 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.923038 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.923092 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.923357 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.923441 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 16:09:48.923717 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.919883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:45721 (local address 127.13.8.65:44961)
0924 16:09:48.920010 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.920015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.920410 (+ 395us) server_negotiation.cc:407] Connection header received
0924 16:09:48.920515 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.920518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.920582 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.920651 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:09:48.920907 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.921541 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.922670 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.923097 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.923213 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.923276 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.923486 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.923576 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:09:48.924121 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.919798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.13.8.67:45721)
0924 16:09:48.920309 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:48.920322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:09:48.920427 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:09:48.920674 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:09:48.920676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:09:48.920693 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:09:48.920795 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.920801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.921648 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.921650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.922531 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:09:48.922541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.923236 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.923239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:09:48.923350 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.923375 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 16:09:48.923450 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 16:09:48.923512 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":120,"spinlock_wait_cycles":896,"thread_start_us":143,"threads_started":1}
I20260924 16:09:48.924299 29152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2847f91f8a0c40d5b508daa44b522e4f" candidate_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" is_pre_election: true
I20260924 16:09:48.924674 29152 raft_consensus.cc:2454] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda6532b1ecc4ebd80e248fd91a15d3a in term 0.
I20260924 16:09:48.924764 29018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2847f91f8a0c40d5b508daa44b522e4f" candidate_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab74bc2dea444552a634f85875452681" is_pre_election: true
I20260924 16:09:48.925065 29018 raft_consensus.cc:2454] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda6532b1ecc4ebd80e248fd91a15d3a in term 0.
I20260924 16:09:48.925232 29219 leader_election.cc:302] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a; no voters:
I20260924 16:09:48.925544 29355 raft_consensus.cc:2781] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:09:48.925634 29355 raft_consensus.cc:482] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:09:48.925697 29355 raft_consensus.cc:3037] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:48.926813 29355 raft_consensus.cc:504] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.927204 29355 leader_election.cc:288] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [CANDIDATE]: Term 1 election: Requested vote from peers ab74bc2dea444552a634f85875452681 (127.13.8.65:44961), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
I20260924 16:09:48.927524 29018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2847f91f8a0c40d5b508daa44b522e4f" candidate_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab74bc2dea444552a634f85875452681"
I20260924 16:09:48.927650 29152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2847f91f8a0c40d5b508daa44b522e4f" candidate_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
I20260924 16:09:48.927670 29018 raft_consensus.cc:3037] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:48.927778 29152 raft_consensus.cc:3037] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:09:48.928700 29018 raft_consensus.cc:2454] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda6532b1ecc4ebd80e248fd91a15d3a in term 1.
I20260924 16:09:48.929023 29152 raft_consensus.cc:2454] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda6532b1ecc4ebd80e248fd91a15d3a in term 1.
I20260924 16:09:48.929034 29220 leader_election.cc:302] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab74bc2dea444552a634f85875452681, bda6532b1ecc4ebd80e248fd91a15d3a; no voters:
I20260924 16:09:48.929216 29355 raft_consensus.cc:2781] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:09:48.929450 29355 raft_consensus.cc:686] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 LEADER]: Becoming Leader. State: Replica: bda6532b1ecc4ebd80e248fd91a15d3a, State: Running, Role: LEADER
I20260924 16:09:48.929787 29355 consensus_queue.cc:237] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:09:48.933290 28873 catalog_manager.cc:5495] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67). New cstate: current_term: 1 leader_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } health_report { overall_health: UNKNOWN } } }
I20260924 16:09:48.962436 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:48.958730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.13.8.67:33069)
0924 16:09:48.958854 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 16:09:48.958857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:09:48.958892 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:09:48.959100 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:09:48.959102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:09:48.959180 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:09:48.959269 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:09:48.959537 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.960024 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.960963 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:09:48.961165 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:09:48.961236 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:09:48.961613 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:09:48.962025 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:09:48.962125 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:09:48.962247 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 16:09:48.962328 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:09:48.968755 29018 raft_consensus.cc:1261] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Refusing update from remote peer bda6532b1ecc4ebd80e248fd91a15d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:48.968756 29152 raft_consensus.cc:1261] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Refusing update from remote peer bda6532b1ecc4ebd80e248fd91a15d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:09:48.969461 29355 consensus_queue.cc:1035] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:48.969696 29359 consensus_queue.cc:1035] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:09:48.977489 29364 mvcc.cc:205] Tried to move back new op lower bound from 7332930310007918592 to 7332930309859422208. Current Snapshot: MvccSnapshot[applied={T|T < 7332930310007918592}]
I20260924 16:09:48.980086 29363 mvcc.cc:205] Tried to move back new op lower bound from 7332930310007918592 to 7332930309859422208. Current Snapshot: MvccSnapshot[applied={T|T < 7332930310007918592}]
I20260924 16:09:48.986819 28872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:48.986987 28872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:09:48.989513 28872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Sending DeleteTablet for 3 replicas of tablet 2847f91f8a0c40d5b508daa44b522e4f
I20260924 16:09:48.990417 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:48120
I20260924 16:09:48.990425 29266 tablet_service.cc:1514] Processing DeleteTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:60820
I20260924 16:09:48.990437 28998 tablet_service.cc:1514] Processing DeleteTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:38650
I20260924 16:09:48.991056 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28935
I20260924 16:09:48.991048 29371 tablet_replica.cc:330] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: stopping tablet replica
I20260924 16:09:48.991369 29371 raft_consensus.cc:2229] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:09:48.991765 29371 raft_consensus.cc:2258] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:48.992136 29373 tablet_replica.cc:330] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: stopping tablet replica
I20260924 16:09:48.992432 29373 raft_consensus.cc:2229] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:09:48.992712 29373 raft_consensus.cc:2258] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:09:48.994235 29371 ts_tablet_manager.cc:1905] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:48.994356 29373 ts_tablet_manager.cc:1905] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:09:48.997114 29373 ts_tablet_manager.cc:1918] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:48.997267 29373 log.cc:1196] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/2847f91f8a0c40d5b508daa44b522e4f
I20260924 16:09:48.997654 29373 ts_tablet_manager.cc:1939] T 2847f91f8a0c40d5b508daa44b522e4f P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting consensus metadata
I20260924 16:09:48.998971 29371 ts_tablet_manager.cc:1918] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:09:48.998977 28857 catalog_manager.cc:4841] TS 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): tablet 2847f91f8a0c40d5b508daa44b522e4f (table test-table [id=8335941660684882a43ad2a6cbf146a7]) successfully deleted
I20260924 16:09:48.999066 29371 log.cc:1196] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/2847f91f8a0c40d5b508daa44b522e4f
I20260924 16:09:48.999398 29371 ts_tablet_manager.cc:1939] T 2847f91f8a0c40d5b508daa44b522e4f P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting consensus metadata
I20260924 16:09:49.000515 28857 catalog_manager.cc:4841] TS bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): tablet 2847f91f8a0c40d5b508daa44b522e4f (table test-table [id=8335941660684882a43ad2a6cbf146a7]) successfully deleted
I20260924 16:09:49.003602 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29069
W20260924 16:09:49.003608 28858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:44961 (error 108)
I20260924 16:09:49.005298 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:09:49.004983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.0.0.1:38666)
0924 16:09:49.005150 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 16:09:49.005213 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260924 16:09:49.005450 28858 catalog_manager.cc:4584] TS ab74bc2dea444552a634f85875452681 (127.13.8.65:44961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2847f91f8a0c40d5b508daa44b522e4f: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44961: connect: Connection refused (error 111)
I20260924 16:09:49.010288 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29203
I20260924 16:09:49.015798 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 28841
I20260924 16:10:09.022722 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--webserver_interface=127.13.8.126
--webserver_port=42331
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:09.138525 29375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:09.138840 29375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:09.138922 29375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:09.143044 29375 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 16:10:09.143131 29375 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 16:10:09.143158 29375 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 16:10:09.143189 29375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:09.143235 29375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:09.143280 29375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:09.143332 29375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:09.143371 29375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:09.148854 29375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=42331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:09.150252 29375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:09.151888 29375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:09.166091 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.166079 29382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.166079 29384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.166143 29375 server_base.cc:1034] running on GCE node
I20260924 16:10:09.167047 29375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:09.168319 29375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:09.169516 29375 hybrid_clock.cc:648] HybridClock initialized: now 1790266209169497 us; error 32 us; skew 500 ppm
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:09.180419 29375 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:10:09.181530 29375 webserver.cc:460] Webserver started at http://127.13.8.126:42331/ using document root <none> and password file <none>
I20260924 16:10:09.182116 29375 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:09.182190 29375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:09.186386 29375 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:10:09.188575 29391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.189800 29375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:09.189929 29375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "29e97d652c4d4503b923ba6c507e2149"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "e3ed4625065fc6c85c8430bbb9131b3c"
server_key_iv: "b3f43bc312387e2435f5853263ea675a"
server_key_version: "encryptionkey@0"
I20260924 16:10:09.190629 29375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:09.228612 29375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:09.232192 29375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:09.232398 29375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:09.240239 29375 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41443
I20260924 16:10:09.240528 29443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41443 every 8 connection(s)
I20260924 16:10:09.241432 29375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:09.246026 29444 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:10:09.247714 29444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap starting.
I20260924 16:10:09.249892 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29375
I20260924 16:10:09.250408 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:44961
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=44815
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:10:09.266323 29444 log.cc:824] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:09.275424 29444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:10:09.275817 29444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap complete.
I20260924 16:10:09.279992 29444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:09.280429 29444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Initialized, Role: FOLLOWER
I20260924 16:10:09.280923 29444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:09.281080 29444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:09.281167 29444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:09.281291 29444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 1 FOLLOWER]: Advancing to term 2
I20260924 16:10:09.283264 29444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:09.283609 29444 leader_election.cc:302] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e97d652c4d4503b923ba6c507e2149; no voters:
I20260924 16:10:09.283933 29444 leader_election.cc:288] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 16:10:09.284109 29448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 16:10:09.284487 29448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 LEADER]: Becoming Leader. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Running, Role: LEADER
I20260924 16:10:09.284870 29448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:09.285218 29444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:09.286368 29449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:10:09.286492 29449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:09.287258 29450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e97d652c4d4503b923ba6c507e2149. Latest consensus state: current_term: 2 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:10:09.287417 29450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:09.289930 29464 catalog_manager.cc:1259] Loaded cluster ID: e3ec742ea41d40c09e6386972eedb34c
I20260924 16:10:09.290043 29464 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: loading cluster ID for follower catalog manager: success
I20260924 16:10:09.291896 29464 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: acquiring CA information for follower catalog manager: success
I20260924 16:10:09.293170 29464 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:10:09.293318 29457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:09.295989 29457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:09.296662 29457 catalog_manager.cc:1259] Loaded cluster ID: e3ec742ea41d40c09e6386972eedb34c
I20260924 16:10:09.296891 29457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:09.297814 29457 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:09.298354 29457 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Loaded TSK: 0
I20260924 16:10:09.309749 29457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Generated new TSK 1
I20260924 16:10:09.310094 29457 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:10:09.370469 29446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:09.370766 29446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:09.370857 29446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:09.374866 29446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:09.374957 29446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:09.375057 29446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:09.381246 29446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:44961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=44815
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:09.382665 29446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:09.383924 29446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:10:09.396234 29446 server_base.cc:1034] running on GCE node
W20260924 16:10:09.396423 29472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.396456 29471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.396736 29474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.397186 29446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:09.397887 29446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:09.399067 29446 hybrid_clock.cc:648] HybridClock initialized: now 1790266209399033 us; error 45 us; skew 500 ppm
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:09.401789 29446 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:10:09.402827 29446 webserver.cc:460] Webserver started at http://127.13.8.65:44815/ using document root <none> and password file <none>
I20260924 16:10:09.403403 29446 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:09.403482 29446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:09.407548 29446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:10:09.409795 29481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.411247 29446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:09.411374 29446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "ab74bc2dea444552a634f85875452681"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "a02470538d42df98c0a79de8f8df80cf"
server_key_iv: "5a6e3e26c494973791d763140fd325ce"
server_key_version: "encryptionkey@0"
I20260924 16:10:09.411935 29446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:09.423125 29446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:09.426499 29446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:09.426707 29446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:09.427361 29446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:09.428530 29488 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 16:10:09.432767 29446 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 16:10:09.432830 29446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260924 16:10:09.432899 29446 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 16:10:09.434659 29446 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 16:10:09.434715 29446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:09.435046 29488 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Bootstrap starting.
I20260924 16:10:09.445614 29446 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:44961
I20260924 16:10:09.445673 29595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:44961 every 8 connection(s)
I20260924 16:10:09.446822 29446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:09.447531 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29446
I20260924 16:10:09.448010 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:40871
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=45633
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:10:09.457827 29488 log.cc:824] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:09.465976 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.449312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:45061 (local address 127.13.8.126:41443)
0924 16:10:09.450508 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.450539 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.451020 (+ 481us) server_negotiation.cc:407] Connection header received
0924 16:10:09.452219 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.452230 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.452599 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.452929 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:09.454078 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.455095 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.456267 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.456740 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.456914 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.460040 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:09.460047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:09.460063 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:09.460097 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:09.462265 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.462905 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.462912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.462917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.463077 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.463451 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.463454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.463456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.463913 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:09.464069 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.464876 (+ 807us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.465183 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":111,"threads_started":1}
I20260924 16:10:09.466023 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.449304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:45061)
0924 16:10:09.450830 (+ 1526us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.450898 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.451742 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.452989 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.452998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.453425 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:09.453892 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.453906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.455266 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.455269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.456137 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.456147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.456877 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.456881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.457080 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.457715 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:09.457733 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:09.459798 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:09.462401 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.462407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.462423 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.462774 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.463177 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.463180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.463182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.463347 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.464121 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:09.464128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:09.464517 (+ 389us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.464729 (+ 212us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.465065 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"thread_start_us":149,"threads_started":1}
I20260924 16:10:09.466812 29488 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:10:09.467252 29488 tablet_bootstrap.cc:492] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Bootstrap complete.
I20260924 16:10:09.468479 29488 ts_tablet_manager.cc:1397] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Time spent bootstrapping tablet: real 0.034s user 0.027s sys 0.004s
I20260924 16:10:09.468972 29596 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:10:09.469206 29596 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:09.469736 29596 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:10:09.471338 29488 raft_consensus.cc:348] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:10:09.471824 29488 raft_consensus.cc:729] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab74bc2dea444552a634f85875452681, State: Initialized, Role: FOLLOWER
I20260924 16:10:09.472301 29488 consensus_queue.cc:260] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } }
I20260924 16:10:09.472548 29408 ts_manager.cc:194] Registered new tserver with Master: ab74bc2dea444552a634f85875452681 (127.13.8.65:44961)
I20260924 16:10:09.473272 29488 ts_tablet_manager.cc:1428] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 16:10:09.476493 29408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:45061
I20260924 16:10:09.478921 29596 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:10:09.482470 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.475802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.0.0.1:51502)
0924 16:10:09.477438 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.477481 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.477607 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.478359 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.478365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.478390 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:09.478509 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.478516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.479872 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.479877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.480975 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.480985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.481972 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.481974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.482083 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.482112 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.482196 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.482360 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"thread_start_us":109,"threads_started":1}
I20260924 16:10:09.482498 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.475860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.13.8.65:44961)
0924 16:10:09.477869 (+ 2009us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.477880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.477902 (+ 22us) server_negotiation.cc:407] Connection header received
0924 16:10:09.477965 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.477971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.478168 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.478340 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:09.478647 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.479735 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.481101 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.481858 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.481969 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.482115 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.482281 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.482378 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1914,"thread_start_us":92,"threads_started":1}
I20260924 16:10:09.483567 29530 tablet_service.cc:1514] Processing DeleteTablet for tablet 2847f91f8a0c40d5b508daa44b522e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:09:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51502
I20260924 16:10:09.484407 29605 tablet_replica.cc:330] stopping tablet replica
I20260924 16:10:09.484730 29605 raft_consensus.cc:2229] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:09.484984 29605 raft_consensus.cc:2258] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:09.486020 29605 ts_tablet_manager.cc:1905] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:09.489550 29605 ts_tablet_manager.cc:1918] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:09.489665 29605 log.cc:1196] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal/wals/2847f91f8a0c40d5b508daa44b522e4f
I20260924 16:10:09.489938 29605 ts_tablet_manager.cc:1939] T 2847f91f8a0c40d5b508daa44b522e4f P ab74bc2dea444552a634f85875452681: Deleting consensus metadata
I20260924 16:10:09.491695 29393 catalog_manager.cc:4841] TS ab74bc2dea444552a634f85875452681 (127.13.8.65:44961): tablet 2847f91f8a0c40d5b508daa44b522e4f (table test-table [id=8335941660684882a43ad2a6cbf146a7]) successfully deleted
W20260924 16:10:09.569515 29598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:09.569764 29598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:09.569809 29598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:09.573865 29598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:09.573922 29598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:09.573979 29598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:09.579131 29598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:40871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=45633
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:09.580185 29598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:09.581213 29598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:09.591077 29610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.591149 29598 server_base.cc:1034] running on GCE node
W20260924 16:10:09.591095 29611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.591405 29613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.591807 29598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:09.592442 29598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:09.593770 29598 hybrid_clock.cc:648] HybridClock initialized: now 1790266209593791 us; error 115 us; skew 500 ppm
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:09.596592 29598 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:10:09.597560 29598 webserver.cc:460] Webserver started at http://127.13.8.66:45633/ using document root <none> and password file <none>
I20260924 16:10:09.598152 29598 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:09.598289 29598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:09.602051 29598 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:09.604174 29620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.605445 29598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:09.605578 29598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "86ac61ebeca58ff2a011600f8cdb6e48"
server_key_iv: "a0ccfae9448bfb1bedd6c21ca0a123d6"
server_key_version: "encryptionkey@0"
I20260924 16:10:09.606139 29598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:09.622267 29598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:09.625133 29598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:09.625337 29598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:09.625952 29598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:09.626833 29598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:09.626933 29598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.627032 29598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:09.627084 29598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.636492 29598 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:40871
I20260924 16:10:09.636523 29733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:40871 every 8 connection(s)
I20260924 16:10:09.637507 29598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:09.645896 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29598
I20260924 16:10:09.646499 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:33069
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=45703
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:09.652704 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.639358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:38799 (local address 127.13.8.126:41443)
0924 16:10:09.639515 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.639518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.640329 (+ 811us) server_negotiation.cc:407] Connection header received
0924 16:10:09.641685 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.641688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.641753 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.641861 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:09.643129 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.643696 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.644886 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.645149 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.645266 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.648402 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:09.648406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:09.648408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:09.648436 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:09.649865 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.650672 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.650676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.650677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.650758 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.651223 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.651227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.651228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.651462 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:09.651544 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.652395 (+ 851us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.652569 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:10:09.654013 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.639726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:38799)
0924 16:10:09.640173 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.640214 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.641453 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.642047 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.642055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.642474 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:09.642973 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.642986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.643846 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.643849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.644758 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.644767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.645322 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.645326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.645496 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.646243 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:09.646271 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:09.648204 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:09.649996 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.650005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.650025 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.650471 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.650868 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.650873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.650878 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.651103 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.651579 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:09.651588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:09.652005 (+ 417us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.652283 (+ 278us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.652728 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
I20260924 16:10:09.655432 29734 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:10:09.655704 29734 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:09.656262 29734 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:10:09.657428 29408 ts_manager.cc:194] Registered new tserver with Master: 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
I20260924 16:10:09.658037 29408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:38799
W20260924 16:10:09.764266 29737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:09.764565 29737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:09.764664 29737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:09.768736 29737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:09.768819 29737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:09.768911 29737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:09.774186 29737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:33069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=45703
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:09.775424 29737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:09.776502 29737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:09.786195 29742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:09.786327 29745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.786463 29737 server_base.cc:1034] running on GCE node
W20260924 16:10:09.786597 29743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:09.787046 29737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:09.787848 29737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:09.789007 29737 hybrid_clock.cc:648] HybridClock initialized: now 1790266209788970 us; error 49 us; skew 500 ppm
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:09.791764 29737 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:10:09.792574 29737 webserver.cc:460] Webserver started at http://127.13.8.67:45703/ using document root <none> and password file <none>
I20260924 16:10:09.793102 29737 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:09.793172 29737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:09.796975 29737 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:09.799171 29752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.800417 29737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:09.800550 29737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "24d97933791052949e9ef12454d75d9d"
server_key_iv: "2dc9f3daf681d02288215bce53c94d81"
server_key_version: "encryptionkey@0"
I20260924 16:10:09.801102 29737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:09.812897 29737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:09.816241 29737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:09.816390 29737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:09.816970 29737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:09.817737 29737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:09.817783 29737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.817822 29737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:09.817843 29737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:09.826031 29737 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:33069
I20260924 16:10:09.826123 29865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:33069 every 8 connection(s)
I20260924 16:10:09.827107 29737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:09.833626 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29737
Sep 24 16:10:09 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:09.841984 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.828943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:54063 (local address 127.13.8.126:41443)
0924 16:10:09.829082 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.829085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.829945 (+ 860us) server_negotiation.cc:407] Connection header received
0924 16:10:09.831224 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.831227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.831274 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.831348 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:09.832609 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.833134 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.834652 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.835003 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.835123 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.837716 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:09.837720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:09.837723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:09.837749 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:09.839622 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.840139 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.840144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.840146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.840238 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.840600 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.840604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.840606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.840856 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:09.840968 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.841661 (+ 693us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.841793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 16:10:09.842353 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.834329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.13.8.126:41443)
0924 16:10:09.834559 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.834563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.834769 (+ 206us) server_negotiation.cc:407] Connection header received
0924 16:10:09.834915 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.834920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.835003 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.835121 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:09.835951 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.836475 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.837383 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.837612 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.837725 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.839119 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:09.839127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:09.839131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:09.839161 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:09.840629 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.841102 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.841105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.841107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.841191 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.841418 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.841421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.841423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.841666 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:09.841872 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.842071 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.842181 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260924 16:10:09.843078 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.829322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:54063)
0924 16:10:09.829791 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.829834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.831032 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.831591 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.831598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.831976 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:09.832465 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.832479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.833266 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.833269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.834497 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.834510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.835147 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.835151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.835426 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.836108 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:09.836129 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:09.837563 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:09.839746 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.839751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.839763 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.840016 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.840336 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:09.840339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:09.840341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:09.840484 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:09.840975 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:09.840989 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:09.841337 (+ 348us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.841594 (+ 257us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.841971 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":123,"threads_started":1}
I20260924 16:10:09.844138 29866 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41443
I20260924 16:10:09.844403 29866 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:09.844900 29866 heartbeater.cc:511] Master 127.13.8.126:41443 requested a full tablet report, sending...
I20260924 16:10:09.845884 29408 ts_manager.cc:194] Registered new tserver with Master: bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069)
I20260924 16:10:09.846486 29408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:54063
I20260924 16:10:09.855871 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:09.859615 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.856488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.13.8.126:41443)
0924 16:10:09.856635 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.856644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.856769 (+ 125us) server_negotiation.cc:407] Connection header received
0924 16:10:09.856905 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.856907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.857001 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.857089 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:09.857293 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.857784 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.858622 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.858831 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.858904 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.859147 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:09.859276 (+ 129us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 16:10:09.859485 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43328: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 16:10:09.859688 29869 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43328: token expired
W20260924 16:10:09.860010 29335 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.8.126:41443)
I20260924 16:10:09.866660 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.860336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.13.8.126:41443)
0924 16:10:09.860443 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.860446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.860457 (+ 11us) server_negotiation.cc:407] Connection header received
0924 16:10:09.860520 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.860522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.860555 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.860616 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:09.861174 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.861642 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.862438 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.862641 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.862718 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.863902 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:09.863905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:09.863907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:09.863927 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:09.865321 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.865665 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.865667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.865668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.865738 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:09.865952 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:09.865955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:09.865956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:09.866139 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:09.866297 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.866467 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.866553 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 16:10:09.868942 29408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:09.872272 29408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:09.880368 29530 tablet_service.cc:1467] Processing CreateTablet for tablet 19d473e21a4848418959dede4ce2b162 (DEFAULT_TABLE table=test-table [id=5c0375043dce4ba99aaf0053a1d67bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:09.880980 29530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d473e21a4848418959dede4ce2b162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:09.883817 29488 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Bootstrap starting.
I20260924 16:10:09.884164 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.878942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33069 (local address 127.0.0.1:39622)
0924 16:10:09.879085 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.879101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.879181 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.880095 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.880098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.880117 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:09.880210 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.880216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.881601 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.881607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.882704 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.882715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.883801 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.883805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.883896 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.883923 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.883974 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.884048 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 16:10:09.884743 29488 tablet_bootstrap.cc:654] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:09.884806 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.879228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:40871 (local address 127.0.0.1:55470)
0924 16:10:09.879722 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.879735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.879815 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.880652 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.880655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.880672 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:09.880760 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.880769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.882147 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.882154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.883295 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.883305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.884441 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.884447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.884550 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.884574 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.884633 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.884686 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":124,"threads_started":1}
I20260924 16:10:09.885099 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.879159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.13.8.67:33069)
0924 16:10:09.879555 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.879569 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.879595 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:10:09.879683 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.879688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.879932 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.880127 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:09.880304 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.881462 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.882848 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.883641 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.884333 (+ 692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.884515 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.884570 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.884687 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":166,"threads_started":1}
I20260924 16:10:09.885519 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.879716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.13.8.66:40871)
0924 16:10:09.880144 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.880155 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.880178 (+ 23us) server_negotiation.cc:407] Connection header received
0924 16:10:09.880260 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.880265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.880464 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.880903 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:09.881077 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.882031 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.883444 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.884276 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.885062 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.885207 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.885259 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.885358 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":116,"threads_started":1}
I20260924 16:10:09.886963 29488 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: No bootstrap required, opened a new log
I20260924 16:10:09.887063 29488 ts_tablet_manager.cc:1397] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 16:10:09.887811 29488 raft_consensus.cc:348] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.887939 29488 raft_consensus.cc:374] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:09.887980 29488 raft_consensus.cc:729] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab74bc2dea444552a634f85875452681, State: Initialized, Role: FOLLOWER
I20260924 16:10:09.888127 29488 consensus_queue.cc:260] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.888366 29488 ts_tablet_manager.cc:1428] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 16:10:09.888465 29668 tablet_service.cc:1467] Processing CreateTablet for tablet 19d473e21a4848418959dede4ce2b162 (DEFAULT_TABLE table=test-table [id=5c0375043dce4ba99aaf0053a1d67bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:09.888460 29800 tablet_service.cc:1467] Processing CreateTablet for tablet 19d473e21a4848418959dede4ce2b162 (DEFAULT_TABLE table=test-table [id=5c0375043dce4ba99aaf0053a1d67bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:09.889540 29668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d473e21a4848418959dede4ce2b162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:09.889601 29800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d473e21a4848418959dede4ce2b162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:09.897032 29880 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Bootstrap starting.
I20260924 16:10:09.896746 29879 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Bootstrap starting.
I20260924 16:10:09.899322 29880 tablet_bootstrap.cc:654] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:09.899804 29879 tablet_bootstrap.cc:654] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:09.900409 29880 log.cc:824] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:09.900458 29879 log.cc:824] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:09.902148 29879 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: No bootstrap required, opened a new log
I20260924 16:10:09.902158 29880 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: No bootstrap required, opened a new log
I20260924 16:10:09.902371 29879 ts_tablet_manager.cc:1397] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:09.902398 29880 ts_tablet_manager.cc:1397] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:09.905499 29880 raft_consensus.cc:348] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.905715 29880 raft_consensus.cc:374] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:09.905789 29880 raft_consensus.cc:729] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda6532b1ecc4ebd80e248fd91a15d3a, State: Initialized, Role: FOLLOWER
I20260924 16:10:09.905679 29879 raft_consensus.cc:348] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.906000 29879 raft_consensus.cc:374] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:09.906065 29879 raft_consensus.cc:729] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 192a4c57b3c14a60ae5be9a8ce9184de, State: Initialized, Role: FOLLOWER
I20260924 16:10:09.906301 29880 consensus_queue.cc:260] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.906526 29879 consensus_queue.cc:260] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.907001 29866 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:10:09.907176 29734 heartbeater.cc:503] Master 127.13.8.126:41443 was elected leader, sending a full tablet report...
I20260924 16:10:09.907395 29880 ts_tablet_manager.cc:1428] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:10:09.908522 29879 ts_tablet_manager.cc:1428] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260924 16:10:09.948288 29597 tablet.cc:2369] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:10:09.959025 29884 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:09.959261 29884 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.960443 29884 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), ab74bc2dea444552a634f85875452681 (127.13.8.65:44961)
I20260924 16:10:09.964159 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.960664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33069 (local address 127.13.8.66:56589)
0924 16:10:09.960823 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.960841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.960932 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.961176 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.961179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.961198 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:09.961316 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.961321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.962050 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.962055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.963116 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.963126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.963747 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.963856 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.963875 (+ 19us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.963957 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.964030 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":42}
I20260924 16:10:09.964159 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.960783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:56589 (local address 127.13.8.67:33069)
0924 16:10:09.960930 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.960935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.960950 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:10:09.961021 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.961024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.961074 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.961159 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:09.961425 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.961926 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963235 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.963629 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963745 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.963806 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.963974 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.964046 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:10:09.964401 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.960787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:34041 (local address 127.13.8.65:44961)
0924 16:10:09.960936 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.960943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.961309 (+ 366us) server_negotiation.cc:407] Connection header received
0924 16:10:09.961461 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.961463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.961515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.961595 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:09.961871 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.962440 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963484 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.963864 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963974 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.964044 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.964231 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.964299 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 16:10:09.964411 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.960672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.13.8.66:34041)
0924 16:10:09.961210 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:09.961222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:09.961344 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:09.961616 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:09.961618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:09.961634 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:09.961762 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.961767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.962575 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.962579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.963360 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:09.963369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.963979 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.963982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:09.964111 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.964128 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 16:10:09.964212 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 16:10:09.964278 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":184,"thread_start_us":167,"threads_started":1}
I20260924 16:10:09.964888 29820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d473e21a4848418959dede4ce2b162" candidate_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" is_pre_election: true
I20260924 16:10:09.965090 29550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d473e21a4848418959dede4ce2b162" candidate_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab74bc2dea444552a634f85875452681" is_pre_election: true
I20260924 16:10:09.965191 29820 raft_consensus.cc:2454] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 192a4c57b3c14a60ae5be9a8ce9184de in term 0.
I20260924 16:10:09.965375 29550 raft_consensus.cc:2454] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 192a4c57b3c14a60ae5be9a8ce9184de in term 0.
I20260924 16:10:09.965715 29621 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a; no voters:
I20260924 16:10:09.965979 29884 raft_consensus.cc:2781] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:09.966060 29884 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:09.966102 29884 raft_consensus.cc:3037] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:09.967190 29884 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.967527 29884 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [CANDIDATE]: Term 1 election: Requested vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), ab74bc2dea444552a634f85875452681 (127.13.8.65:44961)
I20260924 16:10:09.967792 29820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d473e21a4848418959dede4ce2b162" candidate_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
I20260924 16:10:09.967895 29550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d473e21a4848418959dede4ce2b162" candidate_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab74bc2dea444552a634f85875452681"
I20260924 16:10:09.967923 29820 raft_consensus.cc:3037] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:09.968043 29550 raft_consensus.cc:3037] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:09.969260 29550 raft_consensus.cc:2454] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 192a4c57b3c14a60ae5be9a8ce9184de in term 1.
I20260924 16:10:09.969261 29820 raft_consensus.cc:2454] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 192a4c57b3c14a60ae5be9a8ce9184de in term 1.
I20260924 16:10:09.969556 29622 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 192a4c57b3c14a60ae5be9a8ce9184de, ab74bc2dea444552a634f85875452681, bda6532b1ecc4ebd80e248fd91a15d3a; no voters:
I20260924 16:10:09.969763 29884 raft_consensus.cc:2781] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:09.969921 29884 raft_consensus.cc:686] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 LEADER]: Becoming Leader. State: Replica: 192a4c57b3c14a60ae5be9a8ce9184de, State: Running, Role: LEADER
I20260924 16:10:09.970319 29884 consensus_queue.cc:237] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:09.973522 29407 catalog_manager.cc:5495] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de reported cstate change: term changed from 0 to 1, leader changed from <none> to 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66). New cstate: current_term: 1 leader_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:09.985129 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:09.981636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.13.8.66:40871)
0924 16:10:09.981753 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:09.981758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:09.981801 (+ 43us) server_negotiation.cc:407] Connection header received
0924 16:10:09.981983 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:09.981985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:09.982038 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:09.982151 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:09.982392 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.982884 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.983716 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:09.983931 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:09.984037 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:09.984338 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:09.984732 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:09.984864 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:09.984923 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 16:10:09.984975 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:10:09.991055 29550 raft_consensus.cc:1261] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Refusing update from remote peer 192a4c57b3c14a60ae5be9a8ce9184de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:09.991065 29820 raft_consensus.cc:1261] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Refusing update from remote peer 192a4c57b3c14a60ae5be9a8ce9184de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:09.991822 29884 consensus_queue.cc:1035] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:09.992062 29886 consensus_queue.cc:1035] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:10.001634 29890 mvcc.cc:205] Tried to move back new op lower bound from 7332930396116447232 to 7332930396041117696. Current Snapshot: MvccSnapshot[applied={T|T < 7332930396116447232}]
I20260924 16:10:10.006901 29407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:10.007055 29407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:10.009373 29407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Sending DeleteTablet for 3 replicas of tablet 19d473e21a4848418959dede4ce2b162
I20260924 16:10:10.010046 29530 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d473e21a4848418959dede4ce2b162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:10 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:51502
I20260924 16:10:10.010423 29899 tablet_replica.cc:330] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: stopping tablet replica
I20260924 16:10:10.010392 29668 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d473e21a4848418959dede4ce2b162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:10 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:55470
I20260924 16:10:10.010608 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29446
I20260924 16:10:10.010715 29899 raft_consensus.cc:2229] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:10.010895 29899 raft_consensus.cc:2258] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:10.010116 29800 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d473e21a4848418959dede4ce2b162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:10 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:39622
I20260924 16:10:10.011566 29900 tablet_replica.cc:330] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: stopping tablet replica
I20260924 16:10:10.011570 29901 tablet_replica.cc:330] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: stopping tablet replica
I20260924 16:10:10.011909 29900 raft_consensus.cc:2229] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:10.011945 29901 raft_consensus.cc:2229] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:10.012315 29900 raft_consensus.cc:2258] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:10.012362 29901 raft_consensus.cc:2258] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:10.014014 29900 ts_tablet_manager.cc:1905] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:10.014200 29901 ts_tablet_manager.cc:1905] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:10.017076 29901 ts_tablet_manager.cc:1918] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:10.017237 29901 log.cc:1196] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/19d473e21a4848418959dede4ce2b162
I20260924 16:10:10.017506 29900 ts_tablet_manager.cc:1918] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:10.017609 29900 log.cc:1196] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/19d473e21a4848418959dede4ce2b162
I20260924 16:10:10.017789 29901 ts_tablet_manager.cc:1939] T 19d473e21a4848418959dede4ce2b162 P bda6532b1ecc4ebd80e248fd91a15d3a: Deleting consensus metadata
I20260924 16:10:10.017956 29900 ts_tablet_manager.cc:1939] T 19d473e21a4848418959dede4ce2b162 P 192a4c57b3c14a60ae5be9a8ce9184de: Deleting consensus metadata
I20260924 16:10:10.019151 29392 catalog_manager.cc:4841] TS 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): tablet 19d473e21a4848418959dede4ce2b162 (table test-table [id=5c0375043dce4ba99aaf0053a1d67bd9]) successfully deleted
I20260924 16:10:10.019322 29392 catalog_manager.cc:4841] TS bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): tablet 19d473e21a4848418959dede4ce2b162 (table test-table [id=5c0375043dce4ba99aaf0053a1d67bd9]) successfully deleted
W20260924 16:10:10.020457 29393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:44961 (error 108)
I20260924 16:10:10.020990 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29598
I20260924 16:10:10.021901 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:10.021271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:44961 (local address 127.0.0.1:51512)
0924 16:10:10.021771 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:10.021830 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422}
W20260924 16:10:10.022090 29393 catalog_manager.cc:4584] TS ab74bc2dea444552a634f85875452681 (127.13.8.65:44961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19d473e21a4848418959dede4ce2b162: Network error: Client connection negotiation failed: client connection to 127.13.8.65:44961: connect: Connection refused (error 111)
I20260924 16:10:10.028033 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29737
I20260924 16:10:10.034013 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29375
I20260924 16:10:30.041653 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--webserver_interface=127.13.8.126
--webserver_port=42331
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:30.154073 29903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:30.154407 29903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:30.154450 29903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:30.158602 29903 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 16:10:30.158667 29903 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 16:10:30.158684 29903 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 16:10:30.158699 29903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:30.158718 29903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:30.158736 29903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:30.158783 29903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:30.158821 29903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:30.164448 29903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=42331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:30.165706 29903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:30.166865 29903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:10:30.176311 29903 server_base.cc:1034] running on GCE node
W20260924 16:10:30.176409 29908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.176440 29911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.176335 29909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:30.177212 29903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:30.178537 29903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:30.179729 29903 hybrid_clock.cc:648] HybridClock initialized: now 1790266230179691 us; error 48 us; skew 500 ppm
Sep 24 16:10:30 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:30.187552 29903 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:10:30.188591 29903 webserver.cc:460] Webserver started at http://127.13.8.126:42331/ using document root <none> and password file <none>
I20260924 16:10:30.189237 29903 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:30.189363 29903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:30.193398 29903 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:10:30.195809 29918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.197093 29903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:30.197242 29903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "29e97d652c4d4503b923ba6c507e2149"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "e3ed4625065fc6c85c8430bbb9131b3c"
server_key_iv: "b3f43bc312387e2435f5853263ea675a"
server_key_version: "encryptionkey@0"
I20260924 16:10:30.197837 29903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:30.222733 29903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:30.225775 29903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:30.226007 29903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:30.233321 29903 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41443
I20260924 16:10:30.233600 29970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41443 every 8 connection(s)
I20260924 16:10:30.234594 29903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:30.238801 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29903
I20260924 16:10:30.239349 29971 sys_catalog.cc:263] Verifying existing consensus state
I20260924 16:10:30.239331 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:44961
--local_ip_for_outbound_sockets=127.13.8.65
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=44815
--webserver_interface=127.13.8.65
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:10:30.241783 29971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap starting.
I20260924 16:10:30.265278 29971 log.cc:824] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:30.276681 29971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:10:30.277084 29971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Bootstrap complete.
I20260924 16:10:30.281172 29971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:30.281591 29971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Initialized, Role: FOLLOWER
I20260924 16:10:30.282140 29971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:30.282330 29971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:30.282408 29971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:30.282533 29971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 2 FOLLOWER]: Advancing to term 3
I20260924 16:10:30.284505 29971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:30.284842 29971 leader_election.cc:302] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e97d652c4d4503b923ba6c507e2149; no voters:
I20260924 16:10:30.285229 29971 leader_election.cc:288] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 16:10:30.285342 29975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 16:10:30.285609 29975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [term 3 LEADER]: Becoming Leader. State: Replica: 29e97d652c4d4503b923ba6c507e2149, State: Running, Role: LEADER
I20260924 16:10:30.285912 29975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } }
I20260924 16:10:30.286558 29971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:30.286934 29977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e97d652c4d4503b923ba6c507e2149. Latest consensus state: current_term: 3 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:10:30.287072 29977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:30.287309 29976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "29e97d652c4d4503b923ba6c507e2149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e97d652c4d4503b923ba6c507e2149" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41443 } } }
I20260924 16:10:30.287422 29976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:30.287787 29979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:30.293078 29979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:30.293736 29979 catalog_manager.cc:1259] Loaded cluster ID: e3ec742ea41d40c09e6386972eedb34c
I20260924 16:10:30.293828 29979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:30.296366 29979 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:30.297117 29979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Loaded TSK: 0
I20260924 16:10:30.297200 29979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 29e97d652c4d4503b923ba6c507e2149: Loaded TSK: 1
F20260924 16:10:30.297979 29979 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f684ac319ad google::LogMessage::Fail() at ??:0
@ 0x7f684ac3488a google::LogMessage::SendToLog() at ??:0
@ 0x7f684ac314de google::LogMessage::Flush() at ??:0
@ 0x7f684ac32a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f684d8f518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f684d8edf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f685559a7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f685557499c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6855575178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f68555b6df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f68555d582b std::function<>::operator()() at ??:0
@ 0x7f6855574b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6855575f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6855572579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f68555b6640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f684ce833d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f684ce83c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f684ce853f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f684ce74512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f684e4b86db start_thread at ??:0
@ 0x7f684972f61f clone at ??:0
W20260924 16:10:30.400934 29973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:30.401342 29973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:30.401417 29973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:30.407652 29973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:30.407738 29973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:30.407840 29973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:30.416273 29973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:44961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=44815
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:30.417706 29973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:30.419143 29973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:30.430418 30019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.430430 30018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.430763 30021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:30.431406 29973 server_base.cc:1034] running on GCE node
I20260924 16:10:30.431828 29973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:30.432518 29973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:30.433714 29973 hybrid_clock.cc:648] HybridClock initialized: now 1790266230433671 us; error 57 us; skew 500 ppm
Sep 24 16:10:30 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:30.436560 29973 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:10:30.437602 29973 webserver.cc:460] Webserver started at http://127.13.8.65:44815/ using document root <none> and password file <none>
I20260924 16:10:30.438199 29973 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:30.438328 29973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:30.442480 29973 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:10:30.444774 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.446130 29973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:10:30.446286 29973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "ab74bc2dea444552a634f85875452681"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "a02470538d42df98c0a79de8f8df80cf"
server_key_iv: "5a6e3e26c494973791d763140fd325ce"
server_key_version: "encryptionkey@0"
I20260924 16:10:30.446864 29973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:30.484333 29973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:30.487944 29973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:30.488173 29973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:30.489029 29973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:30.490294 30035 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 16:10:30.496793 29973 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 16:10:30.496918 29973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260924 16:10:30.497035 29973 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 16:10:30.499071 29973 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 16:10:30.499147 29973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:30.499511 30035 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Bootstrap starting.
I20260924 16:10:30.510031 29973 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:44961
I20260924 16:10:30.510062 30142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:44961 every 8 connection(s)
I20260924 16:10:30.511059 29973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:30.518167 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 29973
I20260924 16:10:30.518643 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:40871
--local_ip_for_outbound_sockets=127.13.8.66
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=45633
--webserver_interface=127.13.8.66
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:10:30.525695 30035 log.cc:824] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:30.533218 30035 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 16:10:30.533614 30035 tablet_bootstrap.cc:492] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Bootstrap complete.
I20260924 16:10:30.534948 30035 ts_tablet_manager.cc:1397] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.008s
I20260924 16:10:30.537595 30035 raft_consensus.cc:348] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:30.538049 30035 raft_consensus.cc:729] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab74bc2dea444552a634f85875452681, State: Initialized, Role: FOLLOWER
I20260924 16:10:30.538611 30035 consensus_queue.cc:260] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:30.539603 30035 ts_tablet_manager.cc:1428] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260924 16:10:30.567294 30145 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:10:30.513206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:59091)
0924 16:10:30.513707 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.513762 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:30.514676 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:30.566482 (+ 51806us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":156,"threads_started":1}
W20260924 16:10:30.567852 30143 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:41443 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:41443: Client connection negotiation failed: client connection to 127.13.8.126:41443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 16:10:30.568600 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.568336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:58923)
0924 16:10:30.568464 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.568545 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 16:10:30.569229 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.569022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:35895)
0924 16:10:30.569127 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.569173 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260924 16:10:30.569398 30143 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 16:10:30.651232 30146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:30.651544 30146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:30.651631 30146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:30.655655 30146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:30.655740 30146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:30.655833 30146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:30.661046 30146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:40871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=45633
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:30.662115 30146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:30.663185 30146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:30.673650 30153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.673657 30154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.673671 30156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:30.673803 30146 server_base.cc:1034] running on GCE node
I20260924 16:10:30.674384 30146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:30.675271 30146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:30.678591 30146 hybrid_clock.cc:648] HybridClock initialized: now 1790266230678563 us; error 34 us; skew 500 ppm
Sep 24 16:10:30 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:30.681298 30146 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:10:30.682271 30146 webserver.cc:460] Webserver started at http://127.13.8.66:45633/ using document root <none> and password file <none>
I20260924 16:10:30.682875 30146 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:30.682942 30146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:30.687192 30146 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:10:30.689352 30163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.690824 30146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:10:30.690953 30146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "192a4c57b3c14a60ae5be9a8ce9184de"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "86ac61ebeca58ff2a011600f8cdb6e48"
server_key_iv: "a0ccfae9448bfb1bedd6c21ca0a123d6"
server_key_version: "encryptionkey@0"
I20260924 16:10:30.691574 30146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:30.721432 30146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:30.725274 30146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:30.725522 30146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:30.726327 30146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:30.727373 30146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:30.727437 30146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.727514 30146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:30.727557 30146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.737948 30146 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:40871
I20260924 16:10:30.737968 30276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:40871 every 8 connection(s)
I20260924 16:10:30.739084 30146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:30.743323 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.741327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:57549)
0924 16:10:30.741837 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.741956 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":107,"threads_started":1}
W20260924 16:10:30.743860 30277 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:41443 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:41443: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
I20260924 16:10:30.744546 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.744279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:54307)
0924 16:10:30.744433 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.744486 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 16:10:30.745229 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.745036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:42095)
0924 16:10:30.745136 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.745180 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:10:30.745456 30277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 16:10:30.747767 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30146
I20260924 16:10:30.748195 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:33069
--local_ip_for_outbound_sockets=127.13.8.67
--tserver_master_addrs=127.13.8.126:41443
--webserver_port=45703
--webserver_interface=127.13.8.67
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:30.861732 30280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:30.861989 30280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:30.862030 30280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:30.866091 30280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:30.866143 30280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:30.866200 30280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:30.871367 30280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:33069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=45703
--tserver_master_addrs=127.13.8.126:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:30.872310 30280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:30.873258 30280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 16:10:30.882864 30280 server_base.cc:1034] running on GCE node
W20260924 16:10:30.882905 30285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.882912 30286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:30.883016 30288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:30.883565 30280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:30.884187 30280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:30.885353 30280 hybrid_clock.cc:648] HybridClock initialized: now 1790266230885323 us; error 74 us; skew 500 ppm
Sep 24 16:10:30 dist-test-slave-vj6n krb5kdc[28828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:30.888188 30280 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:10:30.889027 30280 webserver.cc:460] Webserver started at http://127.13.8.67:45703/ using document root <none> and password file <none>
I20260924 16:10:30.889564 30280 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:30.889621 30280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:30.893463 30280 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:30.895713 30295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.897076 30280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:10:30.897197 30280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "bda6532b1ecc4ebd80e248fd91a15d3a"
format_stamp: "Formatted at 2026-09-24 16:09:48 on dist-test-slave-vj6n"
server_key: "24d97933791052949e9ef12454d75d9d"
server_key_iv: "2dc9f3daf681d02288215bce53c94d81"
server_key_version: "encryptionkey@0"
I20260924 16:10:30.897735 30280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:30.908291 30280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:30.911792 30280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:30.911996 30280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:30.912621 30280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:30.913478 30280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:30.913525 30280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.913566 30280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:30.913618 30280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:30.923429 30280 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:33069
I20260924 16:10:30.923448 30408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:33069 every 8 connection(s)
I20260924 16:10:30.924496 30280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:30.925397 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30280
I20260924 16:10:30.928992 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.926971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:51811)
0924 16:10:30.927489 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.927610 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":135,"threads_started":1}
W20260924 16:10:30.929520 30409 heartbeater.cc:644] Failed to heartbeat to 127.13.8.126:41443 (0 consecutive failures): Network error: Failed to ping master at 127.13.8.126:41443: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
I20260924 16:10:30.930171 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.929899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:47689)
0924 16:10:30.930056 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.930109 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 16:10:30.931049 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:30.930840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:37313)
0924 16:10:30.930952 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:30.930997 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260924 16:10:30.931300 30409 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 16:10:31.570581 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:31.570000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:42181)
0924 16:10:31.570383 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:31.570453 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":76,"threads_started":1}
I20260924 16:10:31.746572 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:31.746112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:43567)
0924 16:10:31.746413 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:31.746474 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":70,"threads_started":1}
I20260924 16:10:31.932199 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:31.931822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:46963)
0924 16:10:31.932051 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:31.932104 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260924 16:10:32.082015 30416 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:32.082381 30416 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:32.083539 30416 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
Sep 24 16:10:32 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.66@KRBTEST.COM
Sep 24 16:10:32 dist-test-slave-vj6n krb5kdc[28828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266230, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.67@KRBTEST.COM
I20260924 16:10:32.098302 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.084456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36693 (local address 127.13.8.66:40871)
0924 16:10:32.084793 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:32.084813 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:32.084841 (+ 28us) server_negotiation.cc:407] Connection header received
0924 16:10:32.085560 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:32.085567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:32.085869 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:32.086192 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:32.087466 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.088597 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.089817 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.090369 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.090587 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:32.093241 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:32.093251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:32.093272 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:32.093311 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:32.095331 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096186 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:32.096190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:32.096195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:32.096307 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096685 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:32.096687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:32.096688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:32.097093 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:32.097244 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:32.097819 (+ 575us) server_negotiation.cc:299] Negotiation successful
0924 16:10:32.098075 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
I20260924 16:10:32.098318 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.083805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:33069 (local address 127.13.8.65:48977)
0924 16:10:32.084467 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:32.084484 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:32.084608 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:32.086160 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:32.086167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:32.086763 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:32.087274 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:32.087292 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.088603 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.088606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:32.089738 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:32.089756 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.090505 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.090509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:32.090676 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:32.091436 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:32.091448 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:32.093333 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:32.095569 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:32.095571 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:32.095573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:32.096026 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:32.096402 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:32.096407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:32.096557 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:32.097220 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:32.097226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:32.097504 (+ 278us) client_negotiation.cc:769] Sending connection context
0924 16:10:32.097705 (+ 201us) client_negotiation.cc:240] Negotiation successful
0924 16:10:32.098074 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":261,"spinlock_wait_cycles":12032,"thread_start_us":104,"threads_started":1}
I20260924 16:10:32.098362 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.084237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:40871 (local address 127.13.8.65:36693)
0924 16:10:32.084519 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:32.084533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:32.084644 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:32.086167 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:32.086170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:32.086763 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:32.087274 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:32.087290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.088746 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.088749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:32.089701 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:32.089712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.090567 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.090571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:32.090681 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:32.091381 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:32.091399 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:32.092986 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:32.095484 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:32.095493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:32.095511 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:32.096031 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:32.096427 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096429 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:32.096430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:32.096566 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:32.097218 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:32.097225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:32.097503 (+ 278us) client_negotiation.cc:769] Sending connection context
0924 16:10:32.097705 (+ 202us) client_negotiation.cc:240] Negotiation successful
0924 16:10:32.098156 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":216,"spinlock_wait_cycles":91648,"thread_start_us":58,"threads_started":1}
I20260924 16:10:32.098323 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.083997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:48977 (local address 127.13.8.67:33069)
0924 16:10:32.084507 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:32.084541 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:32.084599 (+ 58us) server_negotiation.cc:407] Connection header received
0924 16:10:32.085415 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:32.085422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:32.085784 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:32.086038 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:32.087464 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.088414 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.089881 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:32.090381 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:32.090578 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:32.093520 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:32.093525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:32.093536 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:32.093564 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:32.095454 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096151 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:32.096159 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:32.096165 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:32.096298 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:32.096660 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:32.096664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:32.096669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:32.097094 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:32.097244 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:32.097817 (+ 573us) server_negotiation.cc:299] Negotiation successful
0924 16:10:32.098105 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":157,"threads_started":1}
W20260924 16:10:32.101122 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
W20260924 16:10:32.101397 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
I20260924 16:10:32.101457 30029 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab74bc2dea444552a634f85875452681; no voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a
I20260924 16:10:32.101683 30416 raft_consensus.cc:2726] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:10:32.571709 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.571402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:48823)
0924 16:10:32.571562 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:32.571626 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260924 16:10:32.747632 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.747247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:54441)
0924 16:10:32.747491 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:32.747541 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260924 16:10:32.933563 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:32.933049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:58529)
0924 16:10:32.933353 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:32.933457 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260924 16:10:33.573113 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:33.572653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:54419)
0924 16:10:33.572939 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:33.572999 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260924 16:10:33.748988 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:33.748472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:45359)
0924 16:10:33.748760 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:33.748822 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260924 16:10:33.935108 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:33.934596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:56253)
0924 16:10:33.934902 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:33.934984 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260924 16:10:34.121979 30426 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:34.122154 30426 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:34.122639 30426 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
W20260924 16:10:34.123664 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
W20260924 16:10:34.123804 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
I20260924 16:10:34.123876 30029 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab74bc2dea444552a634f85875452681; no voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a
I20260924 16:10:34.124127 30426 raft_consensus.cc:2726] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:10:34.574374 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:34.573936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:44217)
0924 16:10:34.574205 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:34.574274 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260924 16:10:34.750252 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:34.749769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:58191)
0924 16:10:34.750080 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:34.750139 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260924 16:10:34.936558 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:34.936074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:37583)
0924 16:10:34.936390 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:34.936459 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":90,"threads_started":1}
I20260924 16:10:35.576015 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:35.575480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:44489)
0924 16:10:35.575828 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:35.575912 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":78,"threads_started":1}
I20260924 16:10:35.751513 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:35.751083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:45931)
0924 16:10:35.751343 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:35.751407 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260924 16:10:35.938047 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:35.937573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:35563)
0924 16:10:35.937871 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:35.937947 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260924 16:10:36.577286 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:36.576856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:53487)
0924 16:10:36.577122 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:36.577183 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260924 16:10:36.752713 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:36.752301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:37203)
0924 16:10:36.752562 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:36.752618 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260924 16:10:36.938171 30435 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:36.938354 30435 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:36.938820 30435 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
I20260924 16:10:36.939467 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:36.938940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:44171)
0924 16:10:36.939224 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:36.939312 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
W20260924 16:10:36.939630 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
W20260924 16:10:36.939785 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
I20260924 16:10:36.939842 30029 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab74bc2dea444552a634f85875452681; no voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a
I20260924 16:10:36.940001 30435 raft_consensus.cc:2726] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:10:37.578569 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:37.578134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:34629)
0924 16:10:37.578420 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:37.578473 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":136,"threads_started":1}
I20260924 16:10:37.754172 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:37.753521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:54427)
0924 16:10:37.753968 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:37.754048 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":82,"threads_started":1}
I20260924 16:10:37.940855 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:37.940341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:46683)
0924 16:10:37.940660 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:37.940712 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
I20260924 16:10:38.579782 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:38.579308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:53593)
0924 16:10:38.579610 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:38.579674 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":80,"threads_started":1}
I20260924 16:10:38.755452 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:38.755035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:38453)
0924 16:10:38.755294 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:38.755354 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260924 16:10:38.942102 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:38.941650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:34177)
0924 16:10:38.941950 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:38.942005 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":101,"threads_started":1}
I20260924 16:10:39.019148 30443 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:39.019282 30443 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:39.019968 30443 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
W20260924 16:10:39.020871 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
W20260924 16:10:39.021020 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
I20260924 16:10:39.021061 30029 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab74bc2dea444552a634f85875452681; no voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a
I20260924 16:10:39.021186 30443 raft_consensus.cc:2726] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:10:39.581053 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:39.580636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:36511)
0924 16:10:39.580900 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:39.580955 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260924 16:10:39.756719 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:39.756218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:48945)
0924 16:10:39.756543 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:39.756610 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260924 16:10:39.943318 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:39.942865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:52341)
0924 16:10:39.943158 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:39.943217 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260924 16:10:40.582612 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:40.582048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:45443)
0924 16:10:40.582427 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:40.582501 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":76,"threads_started":1}
I20260924 16:10:40.758024 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:40.757554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:40301)
0924 16:10:40.757847 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:40.757925 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260924 16:10:40.944686 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:40.944178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:35465)
0924 16:10:40.944518 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:40.944582 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":105,"threads_started":1}
I20260924 16:10:41.583926 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:41.583453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:54407)
0924 16:10:41.583766 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:41.583829 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
I20260924 16:10:41.693970 30451 raft_consensus.cc:482] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:41.694108 30451 raft_consensus.cc:504] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda6532b1ecc4ebd80e248fd91a15d3a" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 33069 } } peers { permanent_uuid: "192a4c57b3c14a60ae5be9a8ce9184de" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 40871 } } peers { permanent_uuid: "ab74bc2dea444552a634f85875452681" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 44961 } }
I20260924 16:10:41.694690 30451 leader_election.cc:288] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069), 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871)
W20260924 16:10:41.695591 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bda6532b1ecc4ebd80e248fd91a15d3a (127.13.8.67:33069): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
W20260924 16:10:41.695745 30029 leader_election.cc:341] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 192a4c57b3c14a60ae5be9a8ce9184de (127.13.8.66:40871): Not found: Tablet not found: 19d473e21a4848418959dede4ce2b162
I20260924 16:10:41.695806 30029 leader_election.cc:302] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab74bc2dea444552a634f85875452681; no voters: 192a4c57b3c14a60ae5be9a8ce9184de, bda6532b1ecc4ebd80e248fd91a15d3a
I20260924 16:10:41.695938 30451 raft_consensus.cc:2726] T 19d473e21a4848418959dede4ce2b162 P ab74bc2dea444552a634f85875452681 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 16:10:41.759406 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:41.758924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:43083)
0924 16:10:41.759239 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:41.759305 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":111,"threads_started":1}
I20260924 16:10:41.946031 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:41.945550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:55955)
0924 16:10:41.945849 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:41.945920 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260924 16:10:42.585336 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:42.584850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:58833)
0924 16:10:42.585167 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:42.585231 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260924 16:10:42.760768 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:42.760316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:60097)
0924 16:10:42.760601 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:42.760667 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260924 16:10:42.947769 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:42.947104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:41769)
0924 16:10:42.947475 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:42.947634 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":108,"threads_started":1}
I20260924 16:10:43.586781 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:43.586270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:53723)
0924 16:10:43.586606 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:43.586674 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":80,"threads_started":1}
I20260924 16:10:43.762378 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:43.761823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:46497)
0924 16:10:43.762150 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:43.762221 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":100,"threads_started":1}
I20260924 16:10:43.949218 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:43.948744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:55529)
0924 16:10:43.949047 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:43.949116 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":77,"threads_started":1}
I20260924 16:10:44.588043 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:44.587610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:35985)
0924 16:10:44.587880 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:44.587937 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260924 16:10:44.763504 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:44.763104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:52675)
0924 16:10:44.763351 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:44.763406 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260924 16:10:44.950510 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:44.949974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.67:34001)
0924 16:10:44.950320 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:44.950381 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":81,"threads_started":1}
I20260924 16:10:45.589468 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:45.589054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.65:35167)
0924 16:10:45.589313 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:45.589369 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260924 16:10:45.764955 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:45.764543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41443 (local address 127.13.8.66:54311)
0924 16:10:45.764803 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:45.764860 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.126:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260924 16:10:45.931874 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 29973
I20260924 16:10:45.939684 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30146
I20260924 16:10:45.945767 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30280
2026-09-24T16:10:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59183 ms)
[----------] 30 tests from SecurityITest (244598 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:46 dist-test-slave-vj6n krb5kdc[30467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:46 dist-test-slave-vj6n krb5kdc[30467](info): set up 2 sockets
Sep 24 16:10:46 dist-test-slave-vj6n krb5kdc[30467](info): commencing operation
krb5kdc: starting...
W20260924 16:10:47.006886 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:10:47.134784 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:32999
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:32999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:47.254314 30480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:47.254643 30480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:47.254732 30480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:47.258896 30480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:47.258987 30480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:47.259024 30480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:47.259083 30480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:47.259122 30480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:47.264506 30480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:32999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:32999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:47.265671 30480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:47.266839 30480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:47.275967 30485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:47.276099 30480 server_base.cc:1034] running on GCE node
W20260924 16:10:47.276000 30488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:47.275988 30486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:47.276867 30480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:47.278007 30480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:47.279210 30480 hybrid_clock.cc:648] HybridClock initialized: now 1790266247279191 us; error 34 us; skew 500 ppm
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:47.282076 30480 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:10:47.283013 30480 webserver.cc:460] Webserver started at http://127.13.8.126:33199/ using document root <none> and password file <none>
I20260924 16:10:47.283607 30480 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:47.283697 30480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:47.283937 30480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:47.286036 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "e37ba41e331f4733a3a9c68aab7edb6f"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "f68abbdb22cd56cd292f1fc7105d247e"
server_key_iv: "23a491d7f161f8bdb951e55d09b1806c"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.286720 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "e37ba41e331f4733a3a9c68aab7edb6f"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "f68abbdb22cd56cd292f1fc7105d247e"
server_key_iv: "23a491d7f161f8bdb951e55d09b1806c"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.290575 30480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:47.293329 30495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.294791 30480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:47.294938 30480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "e37ba41e331f4733a3a9c68aab7edb6f"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "f68abbdb22cd56cd292f1fc7105d247e"
server_key_iv: "23a491d7f161f8bdb951e55d09b1806c"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.295064 30480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:47.347062 30480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:47.350035 30480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:47.350325 30480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:47.357681 30480 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:32999
I20260924 16:10:47.357707 30547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:32999 every 8 connection(s)
I20260924 16:10:47.358690 30480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:47.361639 30548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:47.362138 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30480
I20260924 16:10:47.362299 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:47.362659 13345 external_mini_cluster.cc:1428] Setting key dca091f108e77ce7030535ed3a770e54
I20260924 16:10:47.368657 30548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: Bootstrap starting.
I20260924 16:10:47.371403 30548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:47.372295 30548 log.cc:824] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:47.374586 30548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: No bootstrap required, opened a new log
I20260924 16:10:47.378289 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:47.364135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.13.8.126:32999)
0924 16:10:47.364567 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:47.364586 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:47.364631 (+ 45us) server_negotiation.cc:407] Connection header received
0924 16:10:47.365464 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:47.365488 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:47.365943 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:47.366358 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:47.367076 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.368573 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.369489 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.370200 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.370383 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:47.373112 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:47.373120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:47.373140 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:47.373166 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:47.375106 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.375592 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.375597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.375602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.375715 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.376035 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.376038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.376039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.376509 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:47.376677 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:47.377043 (+ 366us) server_negotiation.cc:299] Negotiation successful
0924 16:10:47.377393 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260924 16:10:47.378767 30548 raft_consensus.cc:348] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } }
I20260924 16:10:47.379029 30548 raft_consensus.cc:374] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:47.379112 30548 raft_consensus.cc:729] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37ba41e331f4733a3a9c68aab7edb6f, State: Initialized, Role: FOLLOWER
I20260924 16:10:47.379557 30548 consensus_queue.cc:260] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } }
I20260924 16:10:47.379712 30548 raft_consensus.cc:388] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:47.379799 30548 raft_consensus.cc:482] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:47.379918 30548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:47.380864 30548 raft_consensus.cc:504] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } }
I20260924 16:10:47.381213 30548 leader_election.cc:302] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e37ba41e331f4733a3a9c68aab7edb6f; no voters:
I20260924 16:10:47.381541 30548 leader_election.cc:288] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:47.381683 30553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:47.381939 30553 raft_consensus.cc:686] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [term 1 LEADER]: Becoming Leader. State: Replica: e37ba41e331f4733a3a9c68aab7edb6f, State: Running, Role: LEADER
I20260924 16:10:47.382383 30553 consensus_queue.cc:237] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } }
I20260924 16:10:47.382987 30548 sys_catalog.cc:564] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:47.384248 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e37ba41e331f4733a3a9c68aab7edb6f. Latest consensus state: current_term: 1 leader_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } } }
I20260924 16:10:47.384389 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:47.384757 30554 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37ba41e331f4733a3a9c68aab7edb6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 32999 } } }
I20260924 16:10:47.384933 30554 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f [sys.catalog]: This master's current role is: LEADER
W20260924 16:10:47.387987 30568 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:10:47.388094 30568 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:10:47.388178 30563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:47.388908 30563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:47.394201 30563 catalog_manager.cc:1347] Generated new cluster ID: d906219fa9d64f81a8b6f9a55bf68a3b
I20260924 16:10:47.394325 30563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:47.408828 30563 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:47.409766 30563 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:47.418767 30563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: Generated new TSK 0
I20260924 16:10:47.419497 30563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:10:47.453174 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:47.566394 30574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:47.566679 30574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:47.566751 30574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:47.571061 30574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:47.571121 30574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:47.571213 30574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:47.576592 30574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:47.577693 30574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:47.578799 30574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:47.588753 30582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:47.588817 30579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:47.588753 30580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:47.589205 30574 server_base.cc:1034] running on GCE node
I20260924 16:10:47.589612 30574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:47.590191 30574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:47.591403 30574 hybrid_clock.cc:648] HybridClock initialized: now 1790266247591380 us; error 33 us; skew 500 ppm
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:47.594165 30574 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:10:47.595131 30574 webserver.cc:460] Webserver started at http://127.13.8.65:46227/ using document root <none> and password file <none>
I20260924 16:10:47.595727 30574 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:47.595788 30574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:47.596040 30574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:47.598188 30574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "9567f93020c440788e9a74c0786ca590"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "0950a21367a446091b98b06b0580536c"
server_key_iv: "ba6581f98eb4841113420e631af16adf"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.598827 30574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "9567f93020c440788e9a74c0786ca590"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "0950a21367a446091b98b06b0580536c"
server_key_iv: "ba6581f98eb4841113420e631af16adf"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.602919 30574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:10:47.605634 30589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.607128 30574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 16:10:47.607270 30574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "9567f93020c440788e9a74c0786ca590"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "0950a21367a446091b98b06b0580536c"
server_key_iv: "ba6581f98eb4841113420e631af16adf"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.607390 30574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:47.629233 30574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:47.632686 30574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:47.632895 30574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:47.633527 30574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:47.634505 30574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:47.634558 30574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.634627 30574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:47.634656 30574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.643778 30574 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:45795
I20260924 16:10:47.643801 30702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:45795 every 8 connection(s)
I20260924 16:10:47.644703 30574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:47.650020 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30574
I20260924 16:10:47.650177 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:47.650497 13345 external_mini_cluster.cc:1428] Setting key 237a88394d8e6c2331b29a412faa7946
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:10:47.663640 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:47.646557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:57781 (local address 127.13.8.126:32999)
0924 16:10:47.646737 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:47.646742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:47.647592 (+ 850us) server_negotiation.cc:407] Connection header received
0924 16:10:47.648855 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:47.648858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:47.648907 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:47.648983 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:47.650233 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.650864 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.652229 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.652455 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.652572 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:47.655905 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:47.655908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:47.655910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:47.655970 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:47.658144 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.661571 (+ 3427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.661574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.661575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.661657 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.662089 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.662094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.662096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.662490 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:47.662628 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:47.663379 (+ 751us) server_negotiation.cc:299] Negotiation successful
0924 16:10:47.663509 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:10:47.664945 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:47.646986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32999 (local address 127.13.8.65:57781)
0924 16:10:47.647425 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:47.647469 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:47.648649 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:47.649168 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:47.649175 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:47.649532 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:47.650057 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:47.650071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.651000 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.651004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:47.652098 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:47.652108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.652623 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.652627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:47.652804 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:47.653616 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:47.653638 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:47.655689 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:47.661022 (+ 5333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:47.661030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:47.661061 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:47.661362 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:47.661766 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:47.661769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:47.661771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:47.661941 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:47.662627 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:47.662637 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:47.663059 (+ 422us) client_negotiation.cc:769] Sending connection context
0924 16:10:47.663311 (+ 252us) client_negotiation.cc:240] Negotiation successful
0924 16:10:47.663733 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260924 16:10:47.667114 30703 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32999
I20260924 16:10:47.667399 30703 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:47.667930 30703 heartbeater.cc:511] Master 127.13.8.126:32999 requested a full tablet report, sending...
I20260924 16:10:47.669935 30510 ts_manager.cc:194] Registered new tserver with Master: 9567f93020c440788e9a74c0786ca590 (127.13.8.65:45795)
I20260924 16:10:47.671682 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:57781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:10:47.681023 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:47.795979 30708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:47.796257 30708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:47.796303 30708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:47.800557 30708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:47.800614 30708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:47.800675 30708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:47.805969 30708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:47.807111 30708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:47.808109 30708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:47.818665 30713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:47.818708 30716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:47.818665 30714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:47.818835 30708 server_base.cc:1034] running on GCE node
I20260924 16:10:47.819331 30708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:47.819990 30708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:47.821184 30708 hybrid_clock.cc:648] HybridClock initialized: now 1790266247821155 us; error 51 us; skew 500 ppm
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:47.824064 30708 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:10:47.824885 30708 webserver.cc:460] Webserver started at http://127.13.8.66:46487/ using document root <none> and password file <none>
I20260924 16:10:47.825433 30708 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:47.825491 30708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:47.825659 30708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:47.827764 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "a69c1dc868404eacb7789ffb2590d38e"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "68947918edab84dbae809acb7db1ca43"
server_key_iv: "673b229381712183aed9e0889b2c6363"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.828281 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "a69c1dc868404eacb7789ffb2590d38e"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "68947918edab84dbae809acb7db1ca43"
server_key_iv: "673b229381712183aed9e0889b2c6363"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.831990 30708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:10:47.834404 30723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.835639 30708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:47.835762 30708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "a69c1dc868404eacb7789ffb2590d38e"
format_stamp: "Formatted at 2026-09-24 16:10:47 on dist-test-slave-vj6n"
server_key: "68947918edab84dbae809acb7db1ca43"
server_key_iv: "673b229381712183aed9e0889b2c6363"
server_key_version: "encryptionkey@0"
I20260924 16:10:47.835871 30708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:47.857879 30708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:47.861449 30708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:47.861672 30708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:47.862381 30708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:47.863276 30708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:47.863325 30708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.863389 30708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:47.863422 30708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:47.872752 30708 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:33103
I20260924 16:10:47.872781 30836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:33103 every 8 connection(s)
I20260924 16:10:47.873725 30708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:47.878024 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30708
I20260924 16:10:47.878178 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:47.878477 13345 external_mini_cluster.cc:1428] Setting key 42be5332c781aef184aab0e1579be069
Sep 24 16:10:47 dist-test-slave-vj6n krb5kdc[30467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266247, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:10:47.888945 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:47.875614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:46515 (local address 127.13.8.126:32999)
0924 16:10:47.875753 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:47.875756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:47.876672 (+ 916us) server_negotiation.cc:407] Connection header received
0924 16:10:47.878035 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:47.878043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:47.878117 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:47.878248 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:47.879660 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.880189 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.881385 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.881615 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.881750 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:47.884544 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:47.884548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:47.884552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:47.884581 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:47.886350 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.886930 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.886935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.886938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.887038 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:47.887573 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:47.887578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:47.887580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:47.887793 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:47.887919 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:47.888543 (+ 624us) server_negotiation.cc:299] Negotiation successful
0924 16:10:47.888695 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:10:47.889782 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:47.876036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32999 (local address 127.13.8.66:46515)
0924 16:10:47.876509 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:47.876554 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:47.877770 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:47.878443 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:47.878451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:47.878874 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:47.879508 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:47.879523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.880328 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.880331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:47.881255 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:47.881264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:47.881794 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:47.881798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:47.881969 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:47.882627 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:47.882644 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:47.884377 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:47.886488 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:47.886496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:47.886514 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:47.886804 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:47.887150 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:47.887154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:47.887156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:47.887445 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:47.887928 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:47.887938 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:47.888268 (+ 330us) client_negotiation.cc:769] Sending connection context
0924 16:10:47.888491 (+ 223us) client_negotiation.cc:240] Negotiation successful
0924 16:10:47.888861 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:10:47.892428 30837 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32999
I20260924 16:10:47.892724 30837 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:47.893271 30837 heartbeater.cc:511] Master 127.13.8.126:32999 requested a full tablet report, sending...
I20260924 16:10:47.894438 30510 ts_manager.cc:194] Registered new tserver with Master: a69c1dc868404eacb7789ffb2590d38e (127.13.8.66:33103)
I20260924 16:10:47.895030 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:46515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:10:47.909183 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:48.025353 30842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:48.025645 30842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:48.025691 30842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:48.029956 30842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:48.030033 30842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:48.030098 30842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:48.036442 30842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:32999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:48.037681 30842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:48.038957 30842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:48.049755 30850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:48.049809 30848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:48.049755 30847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:48.049887 30842 server_base.cc:1034] running on GCE node
I20260924 16:10:48.050575 30842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:48.051283 30842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:48.052511 30842 hybrid_clock.cc:648] HybridClock initialized: now 1790266248052466 us; error 37 us; skew 500 ppm
Sep 24 16:10:48 dist-test-slave-vj6n krb5kdc[30467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:48.055518 30842 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:10:48.056443 30842 webserver.cc:460] Webserver started at http://127.13.8.67:44487/ using document root <none> and password file <none>
I20260924 16:10:48.057078 30842 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:48.057174 30842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:48.057422 30842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:48.059669 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "87ac9d1704be42349768bbe52c4afb76"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7b3ff61e1bb396f0c44e4a8a2949f65c"
server_key_iv: "0e8960ac1c68a334cd9e469cc6855004"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.060268 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "87ac9d1704be42349768bbe52c4afb76"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7b3ff61e1bb396f0c44e4a8a2949f65c"
server_key_iv: "0e8960ac1c68a334cd9e469cc6855004"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.064304 30842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:10:48.067063 30857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:48.068405 30842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:48.068588 30842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "87ac9d1704be42349768bbe52c4afb76"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7b3ff61e1bb396f0c44e4a8a2949f65c"
server_key_iv: "0e8960ac1c68a334cd9e469cc6855004"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.068722 30842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:48.091661 30842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:48.095229 30842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:48.095490 30842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:48.096211 30842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:48.097163 30842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:48.097239 30842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:48.097316 30842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:48.097370 30842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:48.107501 30842 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:36491
I20260924 16:10:48.107527 30970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:36491 every 8 connection(s)
I20260924 16:10:48.108465 30842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:48.116170 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 30842
I20260924 16:10:48.116338 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:10:48.116679 13345 external_mini_cluster.cc:1428] Setting key 5115dc343199bcdaee6460a00363dc76
Sep 24 16:10:48 dist-test-slave-vj6n krb5kdc[30467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:48.123093 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.110500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:34877 (local address 127.13.8.126:32999)
0924 16:10:48.110694 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.110700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.111722 (+ 1022us) server_negotiation.cc:407] Connection header received
0924 16:10:48.113004 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.113007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.113066 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.113143 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:48.114507 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.115053 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.116298 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.116633 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.116756 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.119521 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:48.119524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:48.119526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:48.119553 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:48.120852 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:48.121463 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:48.121467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:48.121468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:48.121557 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:48.121930 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:48.121932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:48.121933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:48.122139 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:48.122280 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.122830 (+ 550us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.122945 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 16:10:48.123916 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.110987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:32999 (local address 127.13.8.67:34877)
0924 16:10:48.111555 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.111603 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.112787 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.113328 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.113335 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.113753 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:48.114362 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.114380 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.115181 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.115184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.116134 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.116158 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.116837 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.116841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.117000 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.117669 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:48.117694 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:48.119365 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:48.121005 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:48.121011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:48.121027 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:48.121327 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:48.121670 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:48.121673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:48.121674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:48.121836 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:48.122289 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:48.122295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:48.122583 (+ 288us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.122749 (+ 166us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.123034 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260924 16:10:48.125181 30971 heartbeater.cc:347] Connected to a master server at 127.13.8.126:32999
I20260924 16:10:48.125533 30971 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:48.126106 30971 heartbeater.cc:511] Master 127.13.8.126:32999 requested a full tablet report, sending...
I20260924 16:10:48.127259 30510 ts_manager.cc:194] Registered new tserver with Master: 87ac9d1704be42349768bbe52c4afb76 (127.13.8.67:36491)
I20260924 16:10:48.127849 30510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:34877
I20260924 16:10:48.130880 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:48.140323 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.133006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.13.8.126:32999)
0924 16:10:48.133136 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.133142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.133336 (+ 194us) server_negotiation.cc:407] Connection header received
0924 16:10:48.133472 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.133476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.133528 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.133608 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:48.134407 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.134912 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.135734 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.135948 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.136022 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.137455 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:48.137458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:48.137460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:48.137477 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:48.138771 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:48.139237 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:48.139240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:48.139241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:48.139313 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:48.139531 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:48.139532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:48.139533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:48.139739 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:48.139839 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.140025 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.140193 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 16:10:48.143187 30510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:48.146319 30510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:48.160678 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.154757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:45795 (local address 127.0.0.1:39198)
0924 16:10:48.155147 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.155200 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.155333 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.156813 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.156819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.156842 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:48.157016 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.157024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.158057 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.158063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.159300 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.159310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.160147 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.160150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.160256 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.160300 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.160410 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.160523 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":29,"thread_start_us":95,"threads_started":1}
I20260924 16:10:48.160702 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.155583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.13.8.65:45795)
0924 16:10:48.156314 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.156330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.156358 (+ 28us) server_negotiation.cc:407] Connection header received
0924 16:10:48.156440 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.156446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.156654 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.156973 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:48.157136 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.157927 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.159422 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.160026 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.160150 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.160305 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.160455 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.160553 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":91,"threads_started":1}
I20260924 16:10:48.161634 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.155497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:33103 (local address 127.0.0.1:39144)
0924 16:10:48.156324 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.156339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.156433 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.157066 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.157068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.157084 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:48.157171 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.157177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.158593 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.158596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.159971 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.159981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.161211 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.161214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.161309 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.161332 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.161426 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.161506 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":201,"thread_start_us":136,"threads_started":1}
I20260924 16:10:48.161746 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.155427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.13.8.67:36491)
0924 16:10:48.155808 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.155822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.156041 (+ 219us) server_negotiation.cc:407] Connection header received
0924 16:10:48.156425 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.156430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.156604 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.156765 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:48.157166 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.158418 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.160026 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.160901 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.161008 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.161152 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.161489 (+ 337us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.161596 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":191,"threads_started":1}
I20260924 16:10:48.161688 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.155951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39144 (local address 127.13.8.66:33103)
0924 16:10:48.156270 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.156285 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.156521 (+ 236us) server_negotiation.cc:407] Connection header received
0924 16:10:48.156605 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.156609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.156768 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.156898 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:48.157295 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.158461 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.160077 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.160919 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.161019 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.161143 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.161459 (+ 316us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.161551 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260924 16:10:48.162428 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.154840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:36491 (local address 127.0.0.1:35334)
0924 16:10:48.155930 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.155941 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.156062 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.156754 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.156761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.156783 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:48.157016 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.157024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.158593 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.158596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.159887 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.159899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.161216 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.161220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.161331 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.161353 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.162148 (+ 795us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.162278 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":87,"thread_start_us":90,"threads_started":1}
I20260924 16:10:48.164587 30637 tablet_service.cc:1467] Processing CreateTablet for tablet 646348db421c4af49c9c3a38e3279a80 (DEFAULT_TABLE table=test-table [id=4cdcbc0e02a94c9c815cd8545b7d554c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:48.165197 30905 tablet_service.cc:1467] Processing CreateTablet for tablet 646348db421c4af49c9c3a38e3279a80 (DEFAULT_TABLE table=test-table [id=4cdcbc0e02a94c9c815cd8545b7d554c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:48.165196 30771 tablet_service.cc:1467] Processing CreateTablet for tablet 646348db421c4af49c9c3a38e3279a80 (DEFAULT_TABLE table=test-table [id=4cdcbc0e02a94c9c815cd8545b7d554c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:48.165661 30637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646348db421c4af49c9c3a38e3279a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:48.166239 30771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646348db421c4af49c9c3a38e3279a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:48.166253 30905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646348db421c4af49c9c3a38e3279a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:48.172792 30988 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Bootstrap starting.
I20260924 16:10:48.174031 30989 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Bootstrap starting.
I20260924 16:10:48.174530 30991 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Bootstrap starting.
I20260924 16:10:48.175021 30988 tablet_bootstrap.cc:654] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:48.175757 30988 log.cc:824] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:48.176628 30989 tablet_bootstrap.cc:654] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:48.176973 30991 tablet_bootstrap.cc:654] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:48.177528 30989 log.cc:824] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:48.177736 30991 log.cc:824] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:48.177766 30988 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: No bootstrap required, opened a new log
I20260924 16:10:48.177963 30988 ts_tablet_manager.cc:1397] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:10:48.179623 30989 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: No bootstrap required, opened a new log
I20260924 16:10:48.179626 30991 tablet_bootstrap.cc:492] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: No bootstrap required, opened a new log
I20260924 16:10:48.179808 30991 ts_tablet_manager.cc:1397] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:10:48.179808 30989 ts_tablet_manager.cc:1397] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:10:48.181522 30988 raft_consensus.cc:348] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.181775 30988 raft_consensus.cc:374] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:48.181854 30988 raft_consensus.cc:729] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9567f93020c440788e9a74c0786ca590, State: Initialized, Role: FOLLOWER
I20260924 16:10:48.182386 30988 consensus_queue.cc:260] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.183079 30703 heartbeater.cc:503] Master 127.13.8.126:32999 was elected leader, sending a full tablet report...
I20260924 16:10:48.183681 30989 raft_consensus.cc:348] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.183681 30991 raft_consensus.cc:348] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.184021 30989 raft_consensus.cc:374] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:48.184098 30989 raft_consensus.cc:729] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a69c1dc868404eacb7789ffb2590d38e, State: Initialized, Role: FOLLOWER
I20260924 16:10:48.184202 30991 raft_consensus.cc:374] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:48.184305 30991 raft_consensus.cc:729] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ac9d1704be42349768bbe52c4afb76, State: Initialized, Role: FOLLOWER
I20260924 16:10:48.184598 30989 consensus_queue.cc:260] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.184752 30991 consensus_queue.cc:260] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.184834 30988 ts_tablet_manager.cc:1428] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260924 16:10:48.185544 30971 heartbeater.cc:503] Master 127.13.8.126:32999 was elected leader, sending a full tablet report...
I20260924 16:10:48.185639 30989 ts_tablet_manager.cc:1428] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:10:48.185640 30991 ts_tablet_manager.cc:1428] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:10:48.185959 30837 heartbeater.cc:503] Master 127.13.8.126:32999 was elected leader, sending a full tablet report...
W20260924 16:10:48.359553 30972 tablet.cc:2369] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:10:48.374991 30838 tablet.cc:2369] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:10:48.396185 30704 tablet.cc:2369] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:10:48.470688 30997 raft_consensus.cc:482] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:48.470978 30997 raft_consensus.cc:504] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.472332 30997 leader_election.cc:288] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9567f93020c440788e9a74c0786ca590 (127.13.8.65:45795), a69c1dc868404eacb7789ffb2590d38e (127.13.8.66:33103)
I20260924 16:10:48.476387 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.472687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:47165 (local address 127.13.8.65:45795)
0924 16:10:48.472817 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.472823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.472928 (+ 105us) server_negotiation.cc:407] Connection header received
0924 16:10:48.473077 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.473079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.473126 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.473215 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:48.473602 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.474283 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.475335 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.475815 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.475922 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.475988 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.476196 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.476266 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:10:48.476387 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.472548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:45795 (local address 127.13.8.67:47165)
0924 16:10:48.472815 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.472830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.472941 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.473226 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.473229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.473247 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:48.473422 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.473430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.474399 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.474405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.475204 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.475213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.475936 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.475940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.476059 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.476077 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.476177 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.476250 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260924 16:10:48.476552 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.472763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37223 (local address 127.13.8.66:33103)
0924 16:10:48.472884 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.472890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.473115 (+ 225us) server_negotiation.cc:407] Connection header received
0924 16:10:48.473273 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.473280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.473325 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.473416 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:48.473702 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.474205 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.475383 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.475992 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.476114 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.476187 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.476377 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.476443 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:10:48.476580 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.472630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:33103 (local address 127.13.8.67:37223)
0924 16:10:48.473016 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.473027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:48.473153 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:48.473424 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:48.473427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:48.473462 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:48.473592 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.473598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.474337 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.474344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.475255 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:48.475265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.476117 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.476121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:48.476246 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.476274 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 16:10:48.476361 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 16:10:48.476428 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":63,"thread_start_us":118,"threads_started":1}
I20260924 16:10:48.477149 30657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646348db421c4af49c9c3a38e3279a80" candidate_uuid: "87ac9d1704be42349768bbe52c4afb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567f93020c440788e9a74c0786ca590" is_pre_election: true
I20260924 16:10:48.477288 30791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646348db421c4af49c9c3a38e3279a80" candidate_uuid: "87ac9d1704be42349768bbe52c4afb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a69c1dc868404eacb7789ffb2590d38e" is_pre_election: true
I20260924 16:10:48.477530 30791 raft_consensus.cc:2454] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac9d1704be42349768bbe52c4afb76 in term 0.
I20260924 16:10:48.477510 30657 raft_consensus.cc:2454] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac9d1704be42349768bbe52c4afb76 in term 0.
I20260924 16:10:48.478013 30858 leader_election.cc:302] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87ac9d1704be42349768bbe52c4afb76, 9567f93020c440788e9a74c0786ca590; no voters:
I20260924 16:10:48.478358 30997 raft_consensus.cc:2781] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:48.478437 30997 raft_consensus.cc:482] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:48.478507 30997 raft_consensus.cc:3037] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:48.479573 30997 raft_consensus.cc:504] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.479959 30997 leader_election.cc:288] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [CANDIDATE]: Term 1 election: Requested vote from peers 9567f93020c440788e9a74c0786ca590 (127.13.8.65:45795), a69c1dc868404eacb7789ffb2590d38e (127.13.8.66:33103)
I20260924 16:10:48.480273 30657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646348db421c4af49c9c3a38e3279a80" candidate_uuid: "87ac9d1704be42349768bbe52c4afb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567f93020c440788e9a74c0786ca590"
I20260924 16:10:48.480329 30791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646348db421c4af49c9c3a38e3279a80" candidate_uuid: "87ac9d1704be42349768bbe52c4afb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a69c1dc868404eacb7789ffb2590d38e"
I20260924 16:10:48.480402 30657 raft_consensus.cc:3037] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:48.480465 30791 raft_consensus.cc:3037] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:48.481444 30791 raft_consensus.cc:2454] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac9d1704be42349768bbe52c4afb76 in term 1.
I20260924 16:10:48.481457 30657 raft_consensus.cc:2454] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac9d1704be42349768bbe52c4afb76 in term 1.
I20260924 16:10:48.481755 30859 leader_election.cc:302] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87ac9d1704be42349768bbe52c4afb76, a69c1dc868404eacb7789ffb2590d38e; no voters:
I20260924 16:10:48.481952 30997 raft_consensus.cc:2781] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:48.482183 30997 raft_consensus.cc:686] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 1 LEADER]: Becoming Leader. State: Replica: 87ac9d1704be42349768bbe52c4afb76, State: Running, Role: LEADER
I20260924 16:10:48.482592 30997 consensus_queue.cc:237] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } }
I20260924 16:10:48.486117 30511 catalog_manager.cc:5495] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87ac9d1704be42349768bbe52c4afb76 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "87ac9d1704be42349768bbe52c4afb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87ac9d1704be42349768bbe52c4afb76" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 36491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:48.527271 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.523585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.13.8.67:36491)
0924 16:10:48.523727 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.523733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.523748 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:10:48.523867 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.523870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.523929 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.524055 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:48.524307 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.524822 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.525711 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.525931 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.526043 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.526422 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:48.526861 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:48.527004 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.527064 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.527121 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:10:48.533612 30657 raft_consensus.cc:1261] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 1 FOLLOWER]: Refusing update from remote peer 87ac9d1704be42349768bbe52c4afb76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:48.533636 30791 raft_consensus.cc:1261] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 1 FOLLOWER]: Refusing update from remote peer 87ac9d1704be42349768bbe52c4afb76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:48.534379 30997 consensus_queue.cc:1035] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9567f93020c440788e9a74c0786ca590" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:48.534590 30999 consensus_queue.cc:1035] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a69c1dc868404eacb7789ffb2590d38e" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 33103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:48.544606 31004 mvcc.cc:205] Tried to move back new op lower bound from 7332930553985933312 to 7332930553787531264. Current Snapshot: MvccSnapshot[applied={T|T < 7332930553985933312}]
I20260924 16:10:48.545019 31003 mvcc.cc:205] Tried to move back new op lower bound from 7332930553985933312 to 7332930553787531264. Current Snapshot: MvccSnapshot[applied={T|T < 7332930553985933312}]
I20260924 16:10:48.551007 30511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:48.551186 30511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:48.553714 30511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e37ba41e331f4733a3a9c68aab7edb6f: Sending DeleteTablet for 3 replicas of tablet 646348db421c4af49c9c3a38e3279a80
I20260924 16:10:48.554548 30637 tablet_service.cc:1514] Processing DeleteTablet for tablet 646348db421c4af49c9c3a38e3279a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:39198
I20260924 16:10:48.554600 30905 tablet_service.cc:1514] Processing DeleteTablet for tablet 646348db421c4af49c9c3a38e3279a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:35334
I20260924 16:10:48.554729 30771 tablet_service.cc:1514] Processing DeleteTablet for tablet 646348db421c4af49c9c3a38e3279a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:48 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:39144
I20260924 16:10:48.555109 31011 tablet_replica.cc:330] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: stopping tablet replica
I20260924 16:10:48.555110 31012 tablet_replica.cc:330] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590: stopping tablet replica
I20260924 16:10:48.555420 31012 raft_consensus.cc:2229] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:48.555420 31011 raft_consensus.cc:2229] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:48.555681 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30574
I20260924 16:10:48.555773 31012 raft_consensus.cc:2258] T 646348db421c4af49c9c3a38e3279a80 P 9567f93020c440788e9a74c0786ca590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:48.555761 31011 raft_consensus.cc:2258] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:48.557302 31011 ts_tablet_manager.cc:1905] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:48.558964 31013 tablet_replica.cc:330] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: stopping tablet replica
I20260924 16:10:48.559324 31013 raft_consensus.cc:2229] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:48.559717 31013 raft_consensus.cc:2258] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:48.560057 31011 ts_tablet_manager.cc:1918] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:48.560209 31011 log.cc:1196] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/646348db421c4af49c9c3a38e3279a80
I20260924 16:10:48.560631 31011 ts_tablet_manager.cc:1939] T 646348db421c4af49c9c3a38e3279a80 P 87ac9d1704be42349768bbe52c4afb76: Deleting consensus metadata
I20260924 16:10:48.561681 31013 ts_tablet_manager.cc:1905] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:48.561872 30497 catalog_manager.cc:4841] TS 87ac9d1704be42349768bbe52c4afb76 (127.13.8.67:36491): tablet 646348db421c4af49c9c3a38e3279a80 (table test-table [id=4cdcbc0e02a94c9c815cd8545b7d554c]) successfully deleted
I20260924 16:10:48.563900 31013 ts_tablet_manager.cc:1918] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:48.564016 31013 log.cc:1196] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/646348db421c4af49c9c3a38e3279a80
I20260924 16:10:48.564306 31013 ts_tablet_manager.cc:1939] T 646348db421c4af49c9c3a38e3279a80 P a69c1dc868404eacb7789ffb2590d38e: Deleting consensus metadata
W20260924 16:10:48.564980 30496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:45795 (error 108)
I20260924 16:10:48.565619 30497 catalog_manager.cc:4841] TS a69c1dc868404eacb7789ffb2590d38e (127.13.8.66:33103): tablet 646348db421c4af49c9c3a38e3279a80 (table test-table [id=4cdcbc0e02a94c9c815cd8545b7d554c]) successfully deleted
I20260924 16:10:48.565819 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30708
I20260924 16:10:48.566828 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.566120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:45795 (local address 127.0.0.1:39210)
0924 16:10:48.566674 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:48.566741 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:45795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477}
W20260924 16:10:48.567485 30496 catalog_manager.cc:4584] TS 9567f93020c440788e9a74c0786ca590 (127.13.8.65:45795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 646348db421c4af49c9c3a38e3279a80: Network error: Client connection negotiation failed: client connection to 127.13.8.65:45795: connect: Connection refused (error 111)
I20260924 16:10:48.572913 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30842
I20260924 16:10:48.579073 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 30480
2026-09-24T16:10:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2632 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T16:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:48Z Disabled control of system clock
I20260924 16:10:48.610428 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41365
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:48.724761 31018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:48.725051 31018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:48.725137 31018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:48.729297 31018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:48.729358 31018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:48.729373 31018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:48.729390 31018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:48.729405 31018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:48.734076 31018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:48.735247 31018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:48.736713 31018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:48.745723 31026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:48.745769 31023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:48.745767 31024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:48.746008 31018 server_base.cc:1034] running on GCE node
I20260924 16:10:48.746690 31018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:48.747860 31018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:48.749060 31018 hybrid_clock.cc:648] HybridClock initialized: now 1790266248749040 us; error 32 us; skew 500 ppm
I20260924 16:10:48.751237 31018 webserver.cc:460] Webserver started at http://127.13.8.126:41395/ using document root <none> and password file <none>
I20260924 16:10:48.752002 31018 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:48.752112 31018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:48.752331 31018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:48.754509 31018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "52f415eb93fa40558be58dc9566dd0f7"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7caa7c4d7e8ebed27e0abf51a2addc20"
server_key_iv: "66d968b2e6e286fafbb2ab04757a919a"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.755084 31018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "52f415eb93fa40558be58dc9566dd0f7"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7caa7c4d7e8ebed27e0abf51a2addc20"
server_key_iv: "66d968b2e6e286fafbb2ab04757a919a"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.758944 31018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:48.761526 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:48.762972 31018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:48.763115 31018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "52f415eb93fa40558be58dc9566dd0f7"
format_stamp: "Formatted at 2026-09-24 16:10:48 on dist-test-slave-vj6n"
server_key: "7caa7c4d7e8ebed27e0abf51a2addc20"
server_key_iv: "66d968b2e6e286fafbb2ab04757a919a"
server_key_version: "encryptionkey@0"
I20260924 16:10:48.763234 31018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:48.788117 31018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:48.789019 31018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:48.789259 31018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:48.796857 31084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41365 every 8 connection(s)
I20260924 16:10:48.796854 31018 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41365
I20260924 16:10:48.797959 31018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:48.801265 31085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:48.806631 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31018
I20260924 16:10:48.806769 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:48.807080 13345 external_mini_cluster.cc:1428] Setting key 5680566754a494f85420957b8887f60a
I20260924 16:10:48.807382 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: Bootstrap starting.
I20260924 16:10:48.810055 31085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:48.810876 31085 log.cc:824] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:48.812944 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: No bootstrap required, opened a new log
I20260924 16:10:48.817168 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:48.808518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.13.8.126:41365)
0924 16:10:48.809226 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:48.809241 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:48.809277 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:10:48.809917 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:48.809935 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:48.810469 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:48.810816 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:48.811314 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.812441 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.813388 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:48.813877 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:48.814063 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:48.815006 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:48.815011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:48.815024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:48.815423 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:48.815480 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:48.815504 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:48.815594 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:48.815789 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 16:10:48.816087 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260924 16:10:48.817502 31085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } }
I20260924 16:10:48.817808 31085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:48.817873 31085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f415eb93fa40558be58dc9566dd0f7, State: Initialized, Role: FOLLOWER
I20260924 16:10:48.818400 31085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } }
I20260924 16:10:48.818576 31085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:48.818651 31085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:48.818750 31085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:48.819932 31085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } }
I20260924 16:10:48.820312 31085 leader_election.cc:302] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52f415eb93fa40558be58dc9566dd0f7; no voters:
I20260924 16:10:48.820631 31085 leader_election.cc:288] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:48.820760 31090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:48.821017 31090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [term 1 LEADER]: Becoming Leader. State: Replica: 52f415eb93fa40558be58dc9566dd0f7, State: Running, Role: LEADER
I20260924 16:10:48.821341 31090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } }
I20260924 16:10:48.822032 31085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:48.823204 31092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f415eb93fa40558be58dc9566dd0f7. Latest consensus state: current_term: 1 leader_uuid: "52f415eb93fa40558be58dc9566dd0f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } } }
I20260924 16:10:48.823344 31092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:48.823201 31091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52f415eb93fa40558be58dc9566dd0f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f415eb93fa40558be58dc9566dd0f7" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41365 } } }
I20260924 16:10:48.823635 31091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:48.824430 31099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:48.827716 31099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:48.832929 31099 catalog_manager.cc:1347] Generated new cluster ID: d85dd9f4647e459da7b5e80ab7b8dd07
I20260924 16:10:48.832998 31099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:48.861135 31099 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:48.862285 31099 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:48.871348 31099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: Generated new TSK 0
I20260924 16:10:48.872241 31099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:10:48.885888 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--builtin_ntp_servers=127.13.8.84:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:49.002833 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:49.003109 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:49.003149 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:49.007194 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:49.007248 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:49.007301 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:49.012157 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:49.013278 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:49.014741 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:49.025367 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:49.025432 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.025432 31109 server_base.cc:1034] running on GCE node
W20260924 16:10:49.025444 31117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.026054 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:49.026850 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:49.028012 31109 hybrid_clock.cc:648] HybridClock initialized: now 1790266249027971 us; error 54 us; skew 500 ppm
I20260924 16:10:49.030010 31109 webserver.cc:460] Webserver started at http://127.13.8.65:32857/ using document root <none> and password file <none>
I20260924 16:10:49.030671 31109 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:49.030737 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:49.030910 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:49.033022 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "942ce82c3dcf49a78a70475106858031"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "71e37e839d969ff519a15ac70eb476e4"
server_key_iv: "a568629eb532aab740b33dd37c7f533a"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.033546 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "942ce82c3dcf49a78a70475106858031"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "71e37e839d969ff519a15ac70eb476e4"
server_key_iv: "a568629eb532aab740b33dd37c7f533a"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.037299 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:10:49.039855 31123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.041137 31109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:49.041291 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "942ce82c3dcf49a78a70475106858031"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "71e37e839d969ff519a15ac70eb476e4"
server_key_iv: "a568629eb532aab740b33dd37c7f533a"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.041425 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:49.067359 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:49.068226 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:49.068461 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:49.069178 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:49.070312 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:49.070372 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.070468 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:49.070502 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.080736 31109 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:36913
I20260924 16:10:49.080755 31236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:36913 every 8 connection(s)
I20260924 16:10:49.081768 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:49.082471 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31109
I20260924 16:10:49.082603 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:49.082922 13345 external_mini_cluster.cc:1428] Setting key 5bc954a9b7bcb5df338b70ed249e5cce
I20260924 16:10:49.085173 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--builtin_ntp_servers=127.13.8.84:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:49.093299 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.084043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:59665 (local address 127.13.8.126:41365)
0924 16:10:49.084192 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.084197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.085654 (+ 1457us) server_negotiation.cc:407] Connection header received
0924 16:10:49.087519 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.087525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.087588 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.087700 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:49.088640 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.089576 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.091021 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.091444 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.091589 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.092700 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:49.092706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:49.092710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:49.092730 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:49.092759 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:49.092773 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:49.092860 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.093010 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.093090 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:10:49.094110 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.084623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41365 (local address 127.13.8.65:59665)
0924 16:10:49.085463 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.085522 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.087263 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.087869 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.087876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.087966 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:49.088487 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.088501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.089742 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.089745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.090882 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.090893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.091618 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.091621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.091766 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.092417 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:49.092419 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:49.092463 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:49.092531 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:49.092866 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:49.092874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:49.092878 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.092975 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.093219 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":114,"threads_started":1}
I20260924 16:10:49.095449 31237 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41365
I20260924 16:10:49.095688 31237 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:49.096246 31237 heartbeater.cc:511] Master 127.13.8.126:41365 requested a full tablet report, sending...
I20260924 16:10:49.097759 31049 ts_manager.cc:194] Registered new tserver with Master: 942ce82c3dcf49a78a70475106858031 (127.13.8.65:36913)
I20260924 16:10:49.099042 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.65:59665
W20260924 16:10:49.203166 31240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:49.203440 31240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:49.203480 31240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:49.207918 31240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:49.207988 31240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:49.208051 31240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:49.213433 31240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:49.214684 31240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:49.216305 31240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:49.226938 31246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.227058 31240 server_base.cc:1034] running on GCE node
W20260924 16:10:49.226959 31245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:49.226938 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.227722 31240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:49.228485 31240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:49.229658 31240 hybrid_clock.cc:648] HybridClock initialized: now 1790266249229634 us; error 35 us; skew 500 ppm
I20260924 16:10:49.231671 31240 webserver.cc:460] Webserver started at http://127.13.8.66:43317/ using document root <none> and password file <none>
I20260924 16:10:49.232318 31240 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:49.232383 31240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:49.232664 31240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:49.234987 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "276fbee0ab324bdfbaf6dc0c774b92ca"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "b98969269def4aab342919540cf30f78"
server_key_iv: "ba1539364700d9d30a82ca4efa50b8c6"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.235594 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "276fbee0ab324bdfbaf6dc0c774b92ca"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "b98969269def4aab342919540cf30f78"
server_key_iv: "ba1539364700d9d30a82ca4efa50b8c6"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.239715 31240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:49.242465 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.243731 31240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:49.243855 31240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "276fbee0ab324bdfbaf6dc0c774b92ca"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "b98969269def4aab342919540cf30f78"
server_key_iv: "ba1539364700d9d30a82ca4efa50b8c6"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.243968 31240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:49.256233 31240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:49.256940 31240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:49.257141 31240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:49.257761 31240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:49.258810 31240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:49.258857 31240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.258930 31240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:49.258965 31240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.267973 31240 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:37507
I20260924 16:10:49.268004 31370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:37507 every 8 connection(s)
I20260924 16:10:49.268915 31240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:49.272306 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31240
I20260924 16:10:49.272405 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:49.272691 13345 external_mini_cluster.cc:1428] Setting key 93a3430cb7c560811e03337e26d92552
I20260924 16:10:49.274835 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--builtin_ntp_servers=127.13.8.84:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:49.279301 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.270798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53809 (local address 127.13.8.126:41365)
0924 16:10:49.270934 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.270937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.272065 (+ 1128us) server_negotiation.cc:407] Connection header received
0924 16:10:49.273826 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.273829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.273873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.273944 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:49.275066 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.275631 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.277075 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.277317 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.277433 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.278775 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:49.278778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:49.278780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:49.278793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:49.278813 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:49.278821 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:49.278882 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.279119 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.279189 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:10:49.280613 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.271211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41365 (local address 127.13.8.66:53809)
0924 16:10:49.271913 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.271958 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.273596 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.274165 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.274175 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.274327 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:49.274863 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.274877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.275771 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.275777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.276939 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.276950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.277474 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.277479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.277730 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.278461 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:49.278466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:49.278521 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:49.278617 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:49.278913 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:49.278926 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:49.278936 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.279082 (+ 146us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.279423 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":102,"threads_started":1}
I20260924 16:10:49.282029 31371 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41365
I20260924 16:10:49.282334 31371 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:49.282886 31371 heartbeater.cc:511] Master 127.13.8.126:41365 requested a full tablet report, sending...
I20260924 16:10:49.284008 31049 ts_manager.cc:194] Registered new tserver with Master: 276fbee0ab324bdfbaf6dc0c774b92ca (127.13.8.66:37507)
I20260924 16:10:49.284570 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.66:53809
W20260924 16:10:49.390384 31374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:49.390908 31374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:49.390993 31374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:49.395144 31374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:49.395223 31374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:49.395318 31374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:49.400205 31374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:49.401364 31374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:49.402880 31374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:49.412927 31379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.413038 31374 server_base.cc:1034] running on GCE node
W20260924 16:10:49.412894 31382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:49.412906 31380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:49.413676 31374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:49.414417 31374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:49.415611 31374 hybrid_clock.cc:648] HybridClock initialized: now 1790266249415587 us; error 36 us; skew 500 ppm
I20260924 16:10:49.417421 31374 webserver.cc:460] Webserver started at http://127.13.8.67:39429/ using document root <none> and password file <none>
I20260924 16:10:49.418038 31374 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:49.418141 31374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:49.418408 31374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:49.420492 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "6f22a4036d36481a86f2e0fd357951a9"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "3f4ff15a5a68691e17b655bf4d75b5f4"
server_key_iv: "d9812a32680caebd82ed1494a17d8c8b"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.421077 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "6f22a4036d36481a86f2e0fd357951a9"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "3f4ff15a5a68691e17b655bf4d75b5f4"
server_key_iv: "d9812a32680caebd82ed1494a17d8c8b"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.425143 31374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:10:49.427714 31388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.428996 31374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:49.429142 31374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "6f22a4036d36481a86f2e0fd357951a9"
format_stamp: "Formatted at 2026-09-24 16:10:49 on dist-test-slave-vj6n"
server_key: "3f4ff15a5a68691e17b655bf4d75b5f4"
server_key_iv: "d9812a32680caebd82ed1494a17d8c8b"
server_key_version: "encryptionkey@0"
I20260924 16:10:49.429260 31374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:49.453866 31374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:49.454649 31374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:49.454855 31374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:49.455498 31374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:49.456489 31374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:49.456568 31374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.456638 31374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:49.456691 31374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:49.466312 31374 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:45259
I20260924 16:10:49.466360 31502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:45259 every 8 connection(s)
I20260924 16:10:49.467398 31374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:49.472533 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31374
I20260924 16:10:49.472625 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:10:49.472909 13345 external_mini_cluster.cc:1428] Setting key 1565db70704243343d9c7f95675f9fde
I20260924 16:10:49.477762 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.469391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:54307 (local address 127.13.8.126:41365)
0924 16:10:49.469561 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.469571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.470675 (+ 1104us) server_negotiation.cc:407] Connection header received
0924 16:10:49.472149 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.472153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.472213 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.472335 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:49.473466 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.474254 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.475722 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.475939 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.476039 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.477280 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:49.477284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:49.477286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:49.477298 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:49.477318 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:49.477325 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:49.477394 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.477598 (+ 204us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.477651 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 16:10:49.478755 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.469769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:41365 (local address 127.13.8.67:54307)
0924 16:10:49.470519 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.470566 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.471932 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.472513 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.472521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.472644 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:49.473287 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.473307 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.474408 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.474412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.475611 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.475621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.476115 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.476119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.476285 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.477002 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:49.477004 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:49.477047 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:49.477115 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:49.477416 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:49.477424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:49.477429 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.477578 (+ 149us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.477856 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":107,"threads_started":1}
I20260924 16:10:49.479930 31503 heartbeater.cc:347] Connected to a master server at 127.13.8.126:41365
I20260924 16:10:49.480171 31503 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:49.480696 31503 heartbeater.cc:511] Master 127.13.8.126:41365 requested a full tablet report, sending...
I20260924 16:10:49.481722 31049 ts_manager.cc:194] Registered new tserver with Master: 6f22a4036d36481a86f2e0fd357951a9 (127.13.8.67:45259)
I20260924 16:10:49.482370 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.67:54307
I20260924 16:10:49.486919 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:49.492955 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.488860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.13.8.126:41365)
0924 16:10:49.489001 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.489006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.489191 (+ 185us) server_negotiation.cc:407] Connection header received
0924 16:10:49.489351 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.489353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.489404 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.489489 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:49.489803 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.490389 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.491433 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.491666 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.491780 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.492544 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:49.492549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:49.492551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:49.492563 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:49.492589 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:49.492597 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:49.492674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.492762 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.492816 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:10:49.495712 31049 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:49.498871 31049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:49.513871 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.508693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.13.8.66:37507)
0924 16:10:49.509157 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.509168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.509189 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:10:49.509253 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.509257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.509398 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.509527 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:49.509763 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.510656 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.512655 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513158 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513277 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513477 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.513606 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.513703 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":113,"threads_started":1}
I20260924 16:10:49.513866 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.508303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:37507 (local address 127.0.0.1:51286)
0924 16:10:49.509022 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.509038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.509115 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.509530 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.509541 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.509558 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:49.509642 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.509650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.510771 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.510775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.512509 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.512520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513279 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.513397 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513435 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.513558 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.513708 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":204,"thread_start_us":81,"threads_started":1}
I20260924 16:10:49.514200 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.508246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.13.8.67:45259)
0924 16:10:49.508712 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.508724 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.508959 (+ 235us) server_negotiation.cc:407] Connection header received
0924 16:10:49.509200 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.509206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.509382 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.509526 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:49.509973 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.511273 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.512444 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513224 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513329 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513519 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.513891 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.513994 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":96,"threads_started":1}
I20260924 16:10:49.514501 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.507562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36913 (local address 127.0.0.1:40494)
0924 16:10:49.508125 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.508176 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.508306 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.509165 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.509171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.509195 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:49.509322 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.509327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.510913 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.510917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.512508 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.512520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513647 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.513728 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513767 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.513827 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.513903 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":146,"thread_start_us":73,"threads_started":1}
I20260924 16:10:49.514653 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.507717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:45259 (local address 127.0.0.1:59844)
0924 16:10:49.508865 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.508879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.509041 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.509728 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.509731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.509750 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:49.509847 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.509853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.511393 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.511396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.512312 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.512322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513600 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.513714 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513754 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.514365 (+ 611us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.514456 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
I20260924 16:10:49.515184 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.507722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.13.8.65:36913)
0924 16:10:49.508611 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.508623 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.508649 (+ 26us) server_negotiation.cc:407] Connection header received
0924 16:10:49.508735 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.508741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.508966 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.509450 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:49.509611 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.510804 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.512635 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.513339 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.513426 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.513597 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.514947 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.515058 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":123,"threads_started":1}
I20260924 16:10:49.517225 31304 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 (DEFAULT_TABLE table=test-table [id=54f54e7b339a4f87bfa2e909b436cbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:49.517225 31436 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 (DEFAULT_TABLE table=test-table [id=54f54e7b339a4f87bfa2e909b436cbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:49.518221 31304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cbc94afc4204a43a51d02c6e2a47004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:49.518283 31436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cbc94afc4204a43a51d02c6e2a47004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:49.520562 31171 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 (DEFAULT_TABLE table=test-table [id=54f54e7b339a4f87bfa2e909b436cbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:49.521727 31171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cbc94afc4204a43a51d02c6e2a47004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:49.525388 31520 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Bootstrap starting.
I20260924 16:10:49.526943 31521 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Bootstrap starting.
I20260924 16:10:49.527937 31520 tablet_bootstrap.cc:654] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:49.528623 31522 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Bootstrap starting.
I20260924 16:10:49.529173 31521 tablet_bootstrap.cc:654] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:49.529632 31520 log.cc:824] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:49.529933 31521 log.cc:824] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:49.531157 31522 tablet_bootstrap.cc:654] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:49.531966 31522 log.cc:824] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:49.532013 31521 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: No bootstrap required, opened a new log
I20260924 16:10:49.532197 31520 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: No bootstrap required, opened a new log
I20260924 16:10:49.532209 31521 ts_tablet_manager.cc:1397] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260924 16:10:49.532428 31520 ts_tablet_manager.cc:1397] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260924 16:10:49.533973 31522 tablet_bootstrap.cc:492] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: No bootstrap required, opened a new log
I20260924 16:10:49.534165 31522 ts_tablet_manager.cc:1397] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:10:49.535835 31521 raft_consensus.cc:348] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.536123 31521 raft_consensus.cc:374] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:49.536216 31521 raft_consensus.cc:729] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f22a4036d36481a86f2e0fd357951a9, State: Initialized, Role: FOLLOWER
I20260924 16:10:49.536544 31520 raft_consensus.cc:348] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.536682 31521 consensus_queue.cc:260] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.536795 31520 raft_consensus.cc:374] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:49.536850 31520 raft_consensus.cc:729] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276fbee0ab324bdfbaf6dc0c774b92ca, State: Initialized, Role: FOLLOWER
I20260924 16:10:49.537262 31520 consensus_queue.cc:260] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.537403 31503 heartbeater.cc:503] Master 127.13.8.126:41365 was elected leader, sending a full tablet report...
I20260924 16:10:49.537735 31521 ts_tablet_manager.cc:1428] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260924 16:10:49.538372 31520 ts_tablet_manager.cc:1428] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20260924 16:10:49.538532 31522 raft_consensus.cc:348] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.538714 31371 heartbeater.cc:503] Master 127.13.8.126:41365 was elected leader, sending a full tablet report...
I20260924 16:10:49.538769 31522 raft_consensus.cc:374] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:49.538833 31522 raft_consensus.cc:729] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942ce82c3dcf49a78a70475106858031, State: Initialized, Role: FOLLOWER
I20260924 16:10:49.539235 31522 consensus_queue.cc:260] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.540310 31522 ts_tablet_manager.cc:1428] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:49.540741 31237 heartbeater.cc:503] Master 127.13.8.126:41365 was elected leader, sending a full tablet report...
W20260924 16:10:49.583253 31238 tablet.cc:2369] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:10:49.718509 31504 tablet.cc:2369] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 16:10:49.770162 31372 tablet.cc:2369] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:10:49.775888 31526 raft_consensus.cc:482] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:49.776211 31526 raft_consensus.cc:504] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.777599 31526 leader_election.cc:288] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 942ce82c3dcf49a78a70475106858031 (127.13.8.65:36913), 276fbee0ab324bdfbaf6dc0c774b92ca (127.13.8.66:37507)
I20260924 16:10:49.781536 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.778025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:42753 (local address 127.13.8.66:37507)
0924 16:10:49.778181 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.778185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.778490 (+ 305us) server_negotiation.cc:407] Connection header received
0924 16:10:49.778651 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.778654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.778707 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.778786 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:49.779083 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.779572 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.780579 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.780948 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.781071 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.781142 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.781307 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.781377 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:10:49.781612 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.777953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37547 (local address 127.13.8.65:36913)
0924 16:10:49.778075 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.778080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.778236 (+ 156us) server_negotiation.cc:407] Connection header received
0924 16:10:49.778430 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.778434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.778492 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.778588 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:49.778883 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.779404 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.780569 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.780989 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.781126 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.781192 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.781383 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.781470 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 16:10:49.781759 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.777834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36913 (local address 127.13.8.67:37547)
0924 16:10:49.778098 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.778115 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.778304 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.778603 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.778606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.778624 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:49.778757 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.778764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.779539 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.779543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.780426 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.780436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.781129 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.781133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.781238 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.781268 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.781366 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.781613 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":211}
I20260924 16:10:49.782111 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.777945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:37507 (local address 127.13.8.67:42753)
0924 16:10:49.778391 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.778402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:49.778531 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:49.778807 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:49.778810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:49.778826 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:49.778965 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.778971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.779681 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.779686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.780446 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:49.780455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.781077 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.781081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:49.781176 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.781205 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 16:10:49.781290 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:10:49.781563 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":163,"threads_started":1}
I20260924 16:10:49.782537 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc94afc4204a43a51d02c6e2a47004" candidate_uuid: "6f22a4036d36481a86f2e0fd357951a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942ce82c3dcf49a78a70475106858031" is_pre_election: true
I20260924 16:10:49.782825 31324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc94afc4204a43a51d02c6e2a47004" candidate_uuid: "6f22a4036d36481a86f2e0fd357951a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" is_pre_election: true
I20260924 16:10:49.782892 31191 raft_consensus.cc:2454] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f22a4036d36481a86f2e0fd357951a9 in term 0.
I20260924 16:10:49.783119 31324 raft_consensus.cc:2454] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f22a4036d36481a86f2e0fd357951a9 in term 0.
I20260924 16:10:49.783538 31391 leader_election.cc:302] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f22a4036d36481a86f2e0fd357951a9, 942ce82c3dcf49a78a70475106858031; no voters:
I20260924 16:10:49.783872 31526 raft_consensus.cc:2781] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:49.783970 31526 raft_consensus.cc:482] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:49.784030 31526 raft_consensus.cc:3037] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:49.785110 31526 raft_consensus.cc:504] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.785526 31526 leader_election.cc:288] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [CANDIDATE]: Term 1 election: Requested vote from peers 942ce82c3dcf49a78a70475106858031 (127.13.8.65:36913), 276fbee0ab324bdfbaf6dc0c774b92ca (127.13.8.66:37507)
I20260924 16:10:49.785878 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc94afc4204a43a51d02c6e2a47004" candidate_uuid: "6f22a4036d36481a86f2e0fd357951a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "942ce82c3dcf49a78a70475106858031"
I20260924 16:10:49.785905 31324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc94afc4204a43a51d02c6e2a47004" candidate_uuid: "6f22a4036d36481a86f2e0fd357951a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca"
I20260924 16:10:49.786000 31191 raft_consensus.cc:3037] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:49.786039 31324 raft_consensus.cc:3037] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:49.787246 31191 raft_consensus.cc:2454] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f22a4036d36481a86f2e0fd357951a9 in term 1.
I20260924 16:10:49.787284 31324 raft_consensus.cc:2454] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f22a4036d36481a86f2e0fd357951a9 in term 1.
I20260924 16:10:49.787590 31391 leader_election.cc:302] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f22a4036d36481a86f2e0fd357951a9, 942ce82c3dcf49a78a70475106858031; no voters:
I20260924 16:10:49.787797 31526 raft_consensus.cc:2781] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:49.788053 31526 raft_consensus.cc:686] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 1 LEADER]: Becoming Leader. State: Replica: 6f22a4036d36481a86f2e0fd357951a9, State: Running, Role: LEADER
I20260924 16:10:49.788388 31526 consensus_queue.cc:237] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } }
I20260924 16:10:49.791903 31049 catalog_manager.cc:5495] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f22a4036d36481a86f2e0fd357951a9 (127.13.8.67). New cstate: current_term: 1 leader_uuid: "6f22a4036d36481a86f2e0fd357951a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f22a4036d36481a86f2e0fd357951a9" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 45259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:49.804806 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.801216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.13.8.67:45259)
0924 16:10:49.801356 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:49.801360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:49.801377 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:10:49.801472 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:49.801474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:49.801525 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:49.801661 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:49.801913 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.802486 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.803330 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:49.803556 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:49.803666 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:49.803977 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:49.804387 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:49.804517 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:49.804590 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 16:10:49.804641 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 16:10:49.811285 31324 raft_consensus.cc:1261] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 1 FOLLOWER]: Refusing update from remote peer 6f22a4036d36481a86f2e0fd357951a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:49.811285 31191 raft_consensus.cc:1261] T 3cbc94afc4204a43a51d02c6e2a47004 P 942ce82c3dcf49a78a70475106858031 [term 1 FOLLOWER]: Refusing update from remote peer 6f22a4036d36481a86f2e0fd357951a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:49.812031 31530 consensus_queue.cc:1035] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "942ce82c3dcf49a78a70475106858031" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:49.812285 31526 consensus_queue.cc:1035] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "276fbee0ab324bdfbaf6dc0c774b92ca" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:49.823118 31535 mvcc.cc:205] Tried to move back new op lower bound from 7332930559218593792 to 7332930559135797248. Current Snapshot: MvccSnapshot[applied={T|T < 7332930559218593792}]
I20260924 16:10:49.824353 31534 mvcc.cc:205] Tried to move back new op lower bound from 7332930559218593792 to 7332930559135797248. Current Snapshot: MvccSnapshot[applied={T|T < 7332930559218593792}]
I20260924 16:10:49.830462 31047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:49.830638 31047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:49.833102 31047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52f415eb93fa40558be58dc9566dd0f7: Sending DeleteTablet for 3 replicas of tablet 3cbc94afc4204a43a51d02c6e2a47004
I20260924 16:10:49.834107 31304 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:49 UTC) from {username='slave'} at 127.0.0.1:51286
I20260924 16:10:49.834132 31171 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:49 UTC) from {username='slave'} at 127.0.0.1:40494
I20260924 16:10:49.834694 31543 tablet_replica.cc:330] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: stopping tablet replica
I20260924 16:10:49.835048 31543 raft_consensus.cc:2229] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:49.835122 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31109
I20260924 16:10:49.835781 31436 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc94afc4204a43a51d02c6e2a47004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:49 UTC) from {username='slave'} at 127.0.0.1:59844
I20260924 16:10:49.836549 31545 tablet_replica.cc:330] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: stopping tablet replica
I20260924 16:10:49.836817 31543 raft_consensus.cc:2258] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:49.836846 31545 raft_consensus.cc:2229] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:49.837215 31545 raft_consensus.cc:2258] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:49.838763 31545 ts_tablet_manager.cc:1905] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:49.838840 31543 ts_tablet_manager.cc:1905] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:49.841781 31545 ts_tablet_manager.cc:1918] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:49.841976 31545 log.cc:1196] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/3cbc94afc4204a43a51d02c6e2a47004
I20260924 16:10:49.842327 31543 ts_tablet_manager.cc:1918] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:49.842438 31543 log.cc:1196] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/3cbc94afc4204a43a51d02c6e2a47004
I20260924 16:10:49.842536 31545 ts_tablet_manager.cc:1939] T 3cbc94afc4204a43a51d02c6e2a47004 P 6f22a4036d36481a86f2e0fd357951a9: Deleting consensus metadata
I20260924 16:10:49.842748 31543 ts_tablet_manager.cc:1939] T 3cbc94afc4204a43a51d02c6e2a47004 P 276fbee0ab324bdfbaf6dc0c774b92ca: Deleting consensus metadata
I20260924 16:10:49.843963 31034 catalog_manager.cc:4841] TS 6f22a4036d36481a86f2e0fd357951a9 (127.13.8.67:45259): tablet 3cbc94afc4204a43a51d02c6e2a47004 (table test-table [id=54f54e7b339a4f87bfa2e909b436cbc7]) successfully deleted
I20260924 16:10:49.844193 31034 catalog_manager.cc:4841] TS 276fbee0ab324bdfbaf6dc0c774b92ca (127.13.8.66:37507): tablet 3cbc94afc4204a43a51d02c6e2a47004 (table test-table [id=54f54e7b339a4f87bfa2e909b436cbc7]) successfully deleted
W20260924 16:10:49.844446 31035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:36913 (error 108)
I20260924 16:10:49.844578 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31240
I20260924 16:10:49.851162 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:49.850845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:36913 (local address 127.0.0.1:40508)
0924 16:10:49.851007 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:49.851077 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:36913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260924 16:10:49.851423 31035 catalog_manager.cc:4584] TS 942ce82c3dcf49a78a70475106858031 (127.13.8.65:36913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3cbc94afc4204a43a51d02c6e2a47004: Network error: Client connection negotiation failed: client connection to 127.13.8.65:36913: connect: Connection refused (error 111)
I20260924 16:10:49.854367 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31374
I20260924 16:10:49.860720 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31018
2026-09-24T16:10:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1280 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:49 dist-test-slave-vj6n krb5kdc[31549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:49 dist-test-slave-vj6n krb5kdc[31549](info): set up 2 sockets
Sep 24 16:10:49 dist-test-slave-vj6n krb5kdc[31549](info): commencing operation
krb5kdc: starting...
W20260924 16:10:50.934890 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:10:51.059618 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35513
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:35513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:51.170849 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:51.171133 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:51.171175 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:51.175103 31562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:51.175151 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:51.175170 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:51.175186 31562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:51.175201 31562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:51.180285 31562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:35513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:51.181295 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:51.182397 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:51.190659 31568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.190659 31567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.190804 31562 server_base.cc:1034] running on GCE node
W20260924 16:10:51.190698 31570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.191591 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:51.192664 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:51.193852 31562 hybrid_clock.cc:648] HybridClock initialized: now 1790266251193832 us; error 31 us; skew 500 ppm
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:51.201391 31562 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:10:51.202363 31562 webserver.cc:460] Webserver started at http://127.13.8.126:41791/ using document root <none> and password file <none>
I20260924 16:10:51.202932 31562 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:51.202992 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:51.203159 31562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:51.205215 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "94cc70b0b2184dd3bafa03c1dbace0f3"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "75101da54c789ff184396911b1fefe18"
server_key_iv: "3e87a126a4e6140abbddb74de11e8a5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.205722 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "94cc70b0b2184dd3bafa03c1dbace0f3"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "75101da54c789ff184396911b1fefe18"
server_key_iv: "3e87a126a4e6140abbddb74de11e8a5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.209388 31562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.003s
I20260924 16:10:51.211892 31577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.213198 31562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:10:51.213296 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "94cc70b0b2184dd3bafa03c1dbace0f3"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "75101da54c789ff184396911b1fefe18"
server_key_iv: "3e87a126a4e6140abbddb74de11e8a5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.213379 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:51.228895 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:51.231769 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:51.231914 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:51.239193 31562 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:35513
I20260924 16:10:51.239236 31630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:35513 every 8 connection(s)
I20260924 16:10:51.240192 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:51.243119 31631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:51.245918 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31562
I20260924 16:10:51.246064 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:51.246409 13345 external_mini_cluster.cc:1428] Setting key 5f3a378f6652b5dbae13433b9bd4d432
I20260924 16:10:51.249337 31631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: Bootstrap starting.
I20260924 16:10:51.251607 31631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:51.252509 31631 log.cc:824] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:51.254909 31631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: No bootstrap required, opened a new log
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:51.258417 31631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } }
I20260924 16:10:51.258656 31631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:51.258716 31631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cc70b0b2184dd3bafa03c1dbace0f3, State: Initialized, Role: FOLLOWER
I20260924 16:10:51.259207 31631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } }
I20260924 16:10:51.259361 31631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:51.259433 31631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:51.259564 31631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:51.260417 31631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } }
I20260924 16:10:51.260807 31631 leader_election.cc:302] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94cc70b0b2184dd3bafa03c1dbace0f3; no voters:
I20260924 16:10:51.261142 31631 leader_election.cc:288] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:51.261260 31636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:51.261514 31636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [term 1 LEADER]: Becoming Leader. State: Replica: 94cc70b0b2184dd3bafa03c1dbace0f3, State: Running, Role: LEADER
I20260924 16:10:51.262281 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.247908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.13.8.126:35513)
0924 16:10:51.248344 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:51.248359 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:51.248395 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:10:51.249006 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:51.249022 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:51.249361 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:51.249763 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:51.250602 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.251605 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.252818 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.253470 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.253613 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.256497 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:51.256511 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:51.256525 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:51.256555 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:51.258524 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.259088 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.259097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.259108 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.259229 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.259599 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.259609 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.259612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.260077 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:51.260334 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:51.260626 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 16:10:51.260992 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":131,"threads_started":1}
I20260924 16:10:51.262403 31631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:51.261842 31636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } }
I20260924 16:10:51.264340 31637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } } }
I20260924 16:10:51.264340 31638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94cc70b0b2184dd3bafa03c1dbace0f3. Latest consensus state: current_term: 1 leader_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94cc70b0b2184dd3bafa03c1dbace0f3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 35513 } } }
I20260924 16:10:51.264506 31638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:51.264508 31637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:51.264943 31645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:51.268280 31645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:51.274084 31645 catalog_manager.cc:1347] Generated new cluster ID: bd7c1f79ae75402db46d150ff0514bc6
I20260924 16:10:51.274171 31645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:51.290714 31645 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:51.292016 31645 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:51.297526 31645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: Generated new TSK 0
I20260924 16:10:51.298388 31645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:10:51.333199 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:51.448953 31657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:51.449270 31657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:51.449347 31657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:51.453500 31657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:51.453593 31657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:51.453683 31657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:51.459187 31657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:51.460340 31657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:51.461449 31657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:51.471985 31662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.472041 31665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.471985 31663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.472230 31657 server_base.cc:1034] running on GCE node
I20260924 16:10:51.472744 31657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:51.473488 31657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:51.474681 31657 hybrid_clock.cc:648] HybridClock initialized: now 1790266251474664 us; error 27 us; skew 500 ppm
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:51.477452 31657 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:10:51.478411 31657 webserver.cc:460] Webserver started at http://127.13.8.65:44151/ using document root <none> and password file <none>
I20260924 16:10:51.478991 31657 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:51.479053 31657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:51.479274 31657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:51.481400 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "760eedc493424f0491e1601ad74ad8e2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "047058abcf64b9bb4377889a83afe714"
server_key_iv: "92bd6354f4fb58d06ee3acb800c53a7d"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.481988 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "760eedc493424f0491e1601ad74ad8e2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "047058abcf64b9bb4377889a83afe714"
server_key_iv: "92bd6354f4fb58d06ee3acb800c53a7d"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.485895 31657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260924 16:10:51.488487 31672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.489748 31657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:51.489867 31657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "760eedc493424f0491e1601ad74ad8e2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "047058abcf64b9bb4377889a83afe714"
server_key_iv: "92bd6354f4fb58d06ee3acb800c53a7d"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.489984 31657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:51.501499 31657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:51.504366 31657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:51.504592 31657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:51.505195 31657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:51.506038 31657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:51.506088 31657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.506155 31657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:51.506186 31657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.515576 31657 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:43711
I20260924 16:10:51.515596 31786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:43711 every 8 connection(s)
I20260924 16:10:51.516690 31657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:51.519979 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31657
I20260924 16:10:51.520123 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:51.520473 13345 external_mini_cluster.cc:1428] Setting key 2e5a7281e54e9391695da2b0a985cd3e
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:10:51.532513 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.518771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:45805 (local address 127.13.8.126:35513)
0924 16:10:51.518965 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:51.518970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:51.520243 (+ 1273us) server_negotiation.cc:407] Connection header received
0924 16:10:51.521433 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:51.521437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:51.521501 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:51.521610 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:51.523225 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.523897 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.525111 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.525544 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.525669 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.528715 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:51.528718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:51.528720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:51.528745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:51.530283 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.530905 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.530910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.530911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.531002 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.531381 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.531384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.531385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.531579 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:51.531659 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:51.532257 (+ 598us) server_negotiation.cc:299] Negotiation successful
0924 16:10:51.532384 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 16:10:51.533406 31789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.519233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35513 (local address 127.13.8.65:45805)
0924 16:10:51.520046 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:51.520125 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:51.521189 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:51.521778 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:51.521788 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:51.522292 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:51.523033 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.523070 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.524049 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.524058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.524976 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.524986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.525700 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.525703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.525853 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.526548 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:51.526567 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:51.528531 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:51.530403 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.530413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.530432 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.530790 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.531103 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.531109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.531114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.531293 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.531684 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:51.531689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:51.532022 (+ 333us) client_negotiation.cc:769] Sending connection context
0924 16:10:51.532202 (+ 180us) client_negotiation.cc:240] Negotiation successful
0924 16:10:51.532503 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":136,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:10:51.534808 31787 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35513
I20260924 16:10:51.535095 31787 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:51.535653 31787 heartbeater.cc:511] Master 127.13.8.126:35513 requested a full tablet report, sending...
I20260924 16:10:51.537292 31594 ts_manager.cc:194] Registered new tserver with Master: 760eedc493424f0491e1601ad74ad8e2 (127.13.8.65:43711)
I20260924 16:10:51.538580 31594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:45805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:10:51.550081 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:51.664142 31792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:51.664418 31792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:51.664463 31792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:51.668788 31792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:51.668845 31792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:51.668946 31792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:51.674330 31792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:51.675421 31792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:51.676566 31792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:51.686293 31797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.686300 31800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.686424 31792 server_base.cc:1034] running on GCE node
W20260924 16:10:51.686300 31798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.687026 31792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:51.687674 31792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:51.688877 31792 hybrid_clock.cc:648] HybridClock initialized: now 1790266251688840 us; error 47 us; skew 500 ppm
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:51.691793 31792 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:10:51.692605 31792 webserver.cc:460] Webserver started at http://127.13.8.66:36249/ using document root <none> and password file <none>
I20260924 16:10:51.693233 31792 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:51.693291 31792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:51.693473 31792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:51.695665 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "1753f0bb525d491a89da7e2fe0381ad2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "33a0014107d430827ea4dfba2f812c9c"
server_key_iv: "9f9209a79720b2e7844b641092ed8fbd"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.696189 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "1753f0bb525d491a89da7e2fe0381ad2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "33a0014107d430827ea4dfba2f812c9c"
server_key_iv: "9f9209a79720b2e7844b641092ed8fbd"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.699796 31792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260924 16:10:51.702107 31807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.703423 31792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:51.703534 31792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "1753f0bb525d491a89da7e2fe0381ad2"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "33a0014107d430827ea4dfba2f812c9c"
server_key_iv: "9f9209a79720b2e7844b641092ed8fbd"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.703617 31792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:51.724355 31792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:51.727746 31792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:51.727895 31792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:51.728412 31792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:51.729259 31792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:51.729305 31792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.729346 31792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:51.729368 31792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.738152 31792 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39701
I20260924 16:10:51.738243 31921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39701 every 8 connection(s)
I20260924 16:10:51.739127 31792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:51.746752 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31792
I20260924 16:10:51.746914 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:51.747219 13345 external_mini_cluster.cc:1428] Setting key 198a2b6b2dfe1aa8548ef59005ab06b6
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:51.753065 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.740993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:43929 (local address 127.13.8.126:35513)
0924 16:10:51.741140 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:51.741143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:51.742192 (+ 1049us) server_negotiation.cc:407] Connection header received
0924 16:10:51.743167 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:51.743170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:51.743229 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:51.743330 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:51.744495 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.745031 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.746201 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.746470 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.746592 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.749394 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:51.749398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:51.749401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:51.749427 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:51.750970 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.751514 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.751517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.751519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.751603 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.751951 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.751954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.751956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.752146 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:51.752254 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:51.752784 (+ 530us) server_negotiation.cc:299] Negotiation successful
0924 16:10:51.752904 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:10:51.754019 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.741385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35513 (local address 127.13.8.66:43929)
0924 16:10:51.742043 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:51.742085 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:51.742937 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:51.743517 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:51.743524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:51.743865 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:51.744351 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.744365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.745160 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.745164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.746075 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.746085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.746639 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.746643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.746865 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.747589 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:51.747613 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:51.749213 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:51.751110 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.751117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.751133 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.751397 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.751697 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.751701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.751703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.751847 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.752253 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:51.752262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:51.752518 (+ 256us) client_negotiation.cc:769] Sending connection context
0924 16:10:51.752741 (+ 223us) client_negotiation.cc:240] Negotiation successful
0924 16:10:51.753084 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":119,"threads_started":1}
I20260924 16:10:51.755221 31922 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35513
I20260924 16:10:51.755496 31922 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:51.756130 31922 heartbeater.cc:511] Master 127.13.8.126:35513 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:10:51.757316 31594 ts_manager.cc:194] Registered new tserver with Master: 1753f0bb525d491a89da7e2fe0381ad2 (127.13.8.66:39701)
I20260924 16:10:51.757899 31594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:43929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:10:51.773200 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:51.889201 31927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:51.889446 31927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:51.889513 31927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:51.893831 31927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:51.893888 31927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:51.893981 31927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:51.899591 31927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:35513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:51.900630 31927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:51.901659 31927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:51.911541 31932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.911504 31935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:51.911509 31933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:51.911866 31927 server_base.cc:1034] running on GCE node
I20260924 16:10:51.912259 31927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:51.912911 31927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:51.914105 31927 hybrid_clock.cc:648] HybridClock initialized: now 1790266251914088 us; error 26 us; skew 500 ppm
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:51.917196 31927 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:10:51.918258 31927 webserver.cc:460] Webserver started at http://127.13.8.67:36731/ using document root <none> and password file <none>
I20260924 16:10:51.918903 31927 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:51.919010 31927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:51.919257 31927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:51.921404 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "7b794150ade34e7287bbe8f879d7a260"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "8b245ca4eb3354ba930ac3f562dfc948"
server_key_iv: "9556cee4cd2301aa437d33817320cb5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.922024 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "7b794150ade34e7287bbe8f879d7a260"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "8b245ca4eb3354ba930ac3f562dfc948"
server_key_iv: "9556cee4cd2301aa437d33817320cb5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.925952 31927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260924 16:10:51.928681 31942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.930040 31927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:10:51.930190 31927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "7b794150ade34e7287bbe8f879d7a260"
format_stamp: "Formatted at 2026-09-24 16:10:51 on dist-test-slave-vj6n"
server_key: "8b245ca4eb3354ba930ac3f562dfc948"
server_key_iv: "9556cee4cd2301aa437d33817320cb5a"
server_key_version: "encryptionkey@0"
I20260924 16:10:51.930361 31927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:51.943107 31927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:51.946651 31927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:51.946914 31927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:51.947561 31927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:51.948451 31927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:51.948531 31927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.948604 31927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:51.948647 31927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:51.957487 31927 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:37033
I20260924 16:10:51.957510 32055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:37033 every 8 connection(s)
I20260924 16:10:51.958537 31927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:51.959717 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 31927
I20260924 16:10:51.959849 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:10:51.960145 13345 external_mini_cluster.cc:1428] Setting key a10e768ec1197e90b920e9df48f5e362
Sep 24 16:10:51 dist-test-slave-vj6n krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266251, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:51.971962 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.960434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:34889 (local address 127.13.8.126:35513)
0924 16:10:51.960598 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:51.960603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:51.961826 (+ 1223us) server_negotiation.cc:407] Connection header received
0924 16:10:51.962835 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:51.962839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:51.962898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:51.962982 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:51.964140 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.964789 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.965993 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.966258 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.966362 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.968699 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:51.968703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:51.968705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:51.968730 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:51.970017 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.970552 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.970555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.970556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.970635 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.970937 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.970940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.970941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.971136 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:51.971213 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:51.971718 (+ 505us) server_negotiation.cc:299] Negotiation successful
0924 16:10:51.971834 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 16:10:51.972811 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.960907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:35513 (local address 127.13.8.67:34889)
0924 16:10:51.961671 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:51.961723 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:51.962607 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:51.963180 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:51.963187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:51.963534 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:51.963999 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.964011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.964896 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.964898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.965873 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:51.965883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.966402 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.966405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:51.966549 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.967102 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:51.967118 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:51.968540 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:51.970123 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.970128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.970142 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.970462 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.970718 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:51.970721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:51.970722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:51.970859 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:51.971223 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:51.971228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:51.971486 (+ 258us) client_negotiation.cc:769] Sending connection context
0924 16:10:51.971650 (+ 164us) client_negotiation.cc:240] Negotiation successful
0924 16:10:51.971928 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":138,"threads_started":1}
I20260924 16:10:51.973994 32056 heartbeater.cc:347] Connected to a master server at 127.13.8.126:35513
I20260924 16:10:51.974318 32056 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:51.974825 32056 heartbeater.cc:511] Master 127.13.8.126:35513 requested a full tablet report, sending...
I20260924 16:10:51.975860 31594 ts_manager.cc:194] Registered new tserver with Master: 7b794150ade34e7287bbe8f879d7a260 (127.13.8.67:37033)
I20260924 16:10:51.976483 31594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:34889
I20260924 16:10:51.986610 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:51.997071 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:51.988720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.13.8.126:35513)
0924 16:10:51.988891 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:51.988895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:51.989061 (+ 166us) server_negotiation.cc:407] Connection header received
0924 16:10:51.989228 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:51.989231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:51.989279 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:51.989361 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:51.990086 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.990690 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.991538 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:51.991769 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:51.991876 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:51.993552 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:51.993556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:51.993558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:51.993585 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:51.995059 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.995591 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.995594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.995596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.995678 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:51.996027 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:51.996033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:51.996035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:51.996335 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:51.996495 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:51.996787 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 16:10:51.996893 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 16:10:52.000141 31594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:52.003428 31594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:52.019500 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.012674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39701 (local address 127.0.0.1:49184)
0924 16:10:52.013191 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.013241 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.013374 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.014533 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.014543 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.014571 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:52.014754 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.014762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.016375 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.016382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.017630 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.017641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.018867 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.019017 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.019071 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.019194 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.019296 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":84,"thread_start_us":96,"threads_started":1}
I20260924 16:10:52.019534 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.013525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.13.8.66:39701)
0924 16:10:52.013881 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.013895 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.013923 (+ 28us) server_negotiation.cc:407] Connection header received
0924 16:10:52.014009 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.014019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.014267 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.014522 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:52.014893 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.016128 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.017753 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.018717 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018964 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.019101 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.019251 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.019370 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":154,"threads_started":1}
I20260924 16:10:52.019629 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.012679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:37033 (local address 127.0.0.1:49590)
0924 16:10:52.013984 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.014000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.014159 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.014890 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.014895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.014929 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:52.015032 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.015036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.016119 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.016126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.017790 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.017804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.019166 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.019172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.019266 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.019291 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.019370 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.019440 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":982,"thread_start_us":108,"threads_started":1}
I20260924 16:10:52.019759 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.012813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.13.8.67:37033)
0924 16:10:52.013518 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.013537 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.014087 (+ 550us) server_negotiation.cc:407] Connection header received
0924 16:10:52.014365 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.014372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.014570 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.014692 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:52.015151 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.015955 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018010 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.018852 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018948 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.019101 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.019443 (+ 342us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.019562 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":96,"threads_started":1}
I20260924 16:10:52.021445 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.012833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43711 (local address 127.0.0.1:34672)
0924 16:10:52.013813 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.013825 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.013909 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.014627 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.014630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.014643 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:52.014797 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.014811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.016307 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.016311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.017826 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.017836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.018843 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.021078 (+ 2231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.021105 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.021221 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.021298 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":71,"thread_start_us":100,"threads_started":1}
I20260924 16:10:52.021548 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.013307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.13.8.65:43711)
0924 16:10:52.014171 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.014188 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.014211 (+ 23us) server_negotiation.cc:407] Connection header received
0924 16:10:52.014313 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.014318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.014485 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.014790 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:52.014953 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.016133 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.017984 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.018692 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.018832 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.019029 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.021262 (+ 2233us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.021376 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":87,"threads_started":1}
I20260924 16:10:52.023032 31855 tablet_service.cc:1467] Processing CreateTablet for tablet aad24b9127f74b2eb8917916918b8a4c (DEFAULT_TABLE table=test-table [id=736ed1d5205c4f27a1ea9798568e169f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:52.024314 31855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad24b9127f74b2eb8917916918b8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:52.024415 31720 tablet_service.cc:1467] Processing CreateTablet for tablet aad24b9127f74b2eb8917916918b8a4c (DEFAULT_TABLE table=test-table [id=736ed1d5205c4f27a1ea9798568e169f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:52.025496 31720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad24b9127f74b2eb8917916918b8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:52.027925 31990 tablet_service.cc:1467] Processing CreateTablet for tablet aad24b9127f74b2eb8917916918b8a4c (DEFAULT_TABLE table=test-table [id=736ed1d5205c4f27a1ea9798568e169f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:52.028949 31990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad24b9127f74b2eb8917916918b8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:52.032346 32073 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Bootstrap starting.
I20260924 16:10:52.033591 32074 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: Bootstrap starting.
I20260924 16:10:52.035564 32073 tablet_bootstrap.cc:654] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:52.035919 32074 tablet_bootstrap.cc:654] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:52.036283 32075 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Bootstrap starting.
I20260924 16:10:52.036535 32073 log.cc:824] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:52.036795 32074 log.cc:824] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:52.038533 32073 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: No bootstrap required, opened a new log
I20260924 16:10:52.038576 32075 tablet_bootstrap.cc:654] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:52.038743 32073 ts_tablet_manager.cc:1397] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260924 16:10:52.038798 32074 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: No bootstrap required, opened a new log
I20260924 16:10:52.039037 32074 ts_tablet_manager.cc:1397] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260924 16:10:52.039852 32075 log.cc:824] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:52.042091 32075 tablet_bootstrap.cc:492] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: No bootstrap required, opened a new log
I20260924 16:10:52.042310 32075 ts_tablet_manager.cc:1397] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:10:52.042265 32073 raft_consensus.cc:348] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.042573 32073 raft_consensus.cc:374] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:52.042639 32073 raft_consensus.cc:729] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1753f0bb525d491a89da7e2fe0381ad2, State: Initialized, Role: FOLLOWER
I20260924 16:10:52.043112 32073 consensus_queue.cc:260] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.043834 31922 heartbeater.cc:503] Master 127.13.8.126:35513 was elected leader, sending a full tablet report...
I20260924 16:10:52.043672 32074 raft_consensus.cc:348] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.043913 32074 raft_consensus.cc:374] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:52.043960 32074 raft_consensus.cc:729] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760eedc493424f0491e1601ad74ad8e2, State: Initialized, Role: FOLLOWER
I20260924 16:10:52.044184 32073 ts_tablet_manager.cc:1428] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260924 16:10:52.044420 32074 consensus_queue.cc:260] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.045419 32074 ts_tablet_manager.cc:1428] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:10:52.046523 31787 heartbeater.cc:503] Master 127.13.8.126:35513 was elected leader, sending a full tablet report...
I20260924 16:10:52.046116 32075 raft_consensus.cc:348] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.047037 32075 raft_consensus.cc:374] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:52.047124 32075 raft_consensus.cc:729] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b794150ade34e7287bbe8f879d7a260, State: Initialized, Role: FOLLOWER
I20260924 16:10:52.047572 32075 consensus_queue.cc:260] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.048327 32056 heartbeater.cc:503] Master 127.13.8.126:35513 was elected leader, sending a full tablet report...
I20260924 16:10:52.048684 32075 ts_tablet_manager.cc:1428] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:52.099491 32079 raft_consensus.cc:482] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:52.099790 32079 raft_consensus.cc:504] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.101119 32079 leader_election.cc:288] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b794150ade34e7287bbe8f879d7a260 (127.13.8.67:37033), 760eedc493424f0491e1601ad74ad8e2 (127.13.8.65:43711)
I20260924 16:10:52.105331 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.101398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43711 (local address 127.13.8.66:45615)
0924 16:10:52.101560 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.101580 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.101701 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.102002 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.102005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.102028 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:52.102190 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.102199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.103164 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.103168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.104028 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.104038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.104849 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.104853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.104995 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.105021 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.105107 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.105181 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 16:10:52.105355 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.101584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:45615 (local address 127.13.8.65:43711)
0924 16:10:52.101728 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.101735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.101756 (+ 21us) server_negotiation.cc:407] Connection header received
0924 16:10:52.101821 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.101827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.101883 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.101977 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:52.102356 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.103045 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.104180 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.104721 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.104804 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.104859 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.105145 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.105222 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 16:10:52.105508 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.101655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:52321 (local address 127.13.8.67:37033)
0924 16:10:52.101790 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.101796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.102133 (+ 337us) server_negotiation.cc:407] Connection header received
0924 16:10:52.102271 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.102276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.102340 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.102407 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:52.102730 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.103279 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.104435 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.104906 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.105016 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.105083 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.105308 (+ 225us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.105394 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 16:10:52.105891 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.101398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:37033 (local address 127.13.8.66:52321)
0924 16:10:52.102021 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.102036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.102146 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.102454 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.102457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.102478 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:52.102608 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.102616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.103412 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.103414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.104303 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.104313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.105049 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.105052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.105172 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.105195 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.105270 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.105333 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":145,"thread_start_us":126,"threads_started":1}
I20260924 16:10:52.106149 31741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad24b9127f74b2eb8917916918b8a4c" candidate_uuid: "1753f0bb525d491a89da7e2fe0381ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760eedc493424f0491e1601ad74ad8e2" is_pre_election: true
I20260924 16:10:52.106549 31741 raft_consensus.cc:2454] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1753f0bb525d491a89da7e2fe0381ad2 in term 0.
I20260924 16:10:52.106580 32010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad24b9127f74b2eb8917916918b8a4c" candidate_uuid: "1753f0bb525d491a89da7e2fe0381ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b794150ade34e7287bbe8f879d7a260" is_pre_election: true
I20260924 16:10:52.106873 32010 raft_consensus.cc:2454] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1753f0bb525d491a89da7e2fe0381ad2 in term 0.
I20260924 16:10:52.107120 31809 leader_election.cc:302] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1753f0bb525d491a89da7e2fe0381ad2, 760eedc493424f0491e1601ad74ad8e2; no voters:
I20260924 16:10:52.107460 32079 raft_consensus.cc:2781] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:52.107564 32079 raft_consensus.cc:482] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:52.107652 32079 raft_consensus.cc:3037] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:52.108711 32079 raft_consensus.cc:504] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.109112 32079 leader_election.cc:288] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [CANDIDATE]: Term 1 election: Requested vote from peers 7b794150ade34e7287bbe8f879d7a260 (127.13.8.67:37033), 760eedc493424f0491e1601ad74ad8e2 (127.13.8.65:43711)
I20260924 16:10:52.109452 32010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad24b9127f74b2eb8917916918b8a4c" candidate_uuid: "1753f0bb525d491a89da7e2fe0381ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b794150ade34e7287bbe8f879d7a260"
I20260924 16:10:52.109485 31741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad24b9127f74b2eb8917916918b8a4c" candidate_uuid: "1753f0bb525d491a89da7e2fe0381ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760eedc493424f0491e1601ad74ad8e2"
I20260924 16:10:52.109604 32010 raft_consensus.cc:3037] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:52.109622 31741 raft_consensus.cc:3037] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:52.110983 31741 raft_consensus.cc:2454] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1753f0bb525d491a89da7e2fe0381ad2 in term 1.
I20260924 16:10:52.110996 32010 raft_consensus.cc:2454] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1753f0bb525d491a89da7e2fe0381ad2 in term 1.
I20260924 16:10:52.111364 31811 leader_election.cc:302] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1753f0bb525d491a89da7e2fe0381ad2, 760eedc493424f0491e1601ad74ad8e2, 7b794150ade34e7287bbe8f879d7a260; no voters:
I20260924 16:10:52.111549 32079 raft_consensus.cc:2781] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:52.111768 32079 raft_consensus.cc:686] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 1753f0bb525d491a89da7e2fe0381ad2, State: Running, Role: LEADER
I20260924 16:10:52.112138 32079 consensus_queue.cc:237] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } }
I20260924 16:10:52.115418 31595 catalog_manager.cc:5495] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1753f0bb525d491a89da7e2fe0381ad2 (127.13.8.66). New cstate: current_term: 1 leader_uuid: "1753f0bb525d491a89da7e2fe0381ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1753f0bb525d491a89da7e2fe0381ad2" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:52.144335 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.140556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.13.8.66:39701)
0924 16:10:52.140725 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.140732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.140748 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:10:52.140835 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.140837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.140893 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.141028 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:52.141288 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.141816 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.142723 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.142942 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.143061 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.143398 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:52.143877 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:52.144067 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.144133 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.144177 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 16:10:52.150748 31741 raft_consensus.cc:1261] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 1 FOLLOWER]: Refusing update from remote peer 1753f0bb525d491a89da7e2fe0381ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:52.150748 32010 raft_consensus.cc:1261] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 1 FOLLOWER]: Refusing update from remote peer 1753f0bb525d491a89da7e2fe0381ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:52.151552 32083 consensus_queue.cc:1035] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b794150ade34e7287bbe8f879d7a260" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:52.151743 32079 consensus_queue.cc:1035] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "760eedc493424f0491e1601ad74ad8e2" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 43711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:52.159204 32086 mvcc.cc:205] Tried to move back new op lower bound from 7332930568801468416 to 7332930568653881344. Current Snapshot: MvccSnapshot[applied={T|T < 7332930568801468416}]
I20260924 16:10:52.161269 32089 mvcc.cc:205] Tried to move back new op lower bound from 7332930568801468416 to 7332930568653881344. Current Snapshot: MvccSnapshot[applied={T|T < 7332930568801468416}]
I20260924 16:10:52.169729 31594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:52.169917 31594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:52.172474 31594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94cc70b0b2184dd3bafa03c1dbace0f3: Sending DeleteTablet for 3 replicas of tablet aad24b9127f74b2eb8917916918b8a4c
I20260924 16:10:52.173295 31855 tablet_service.cc:1514] Processing DeleteTablet for tablet aad24b9127f74b2eb8917916918b8a4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:52 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:49184
I20260924 16:10:52.173295 31990 tablet_service.cc:1514] Processing DeleteTablet for tablet aad24b9127f74b2eb8917916918b8a4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:52 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:49590
I20260924 16:10:52.173341 31720 tablet_service.cc:1514] Processing DeleteTablet for tablet aad24b9127f74b2eb8917916918b8a4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:52 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:34672
I20260924 16:10:52.173972 32097 tablet_replica.cc:330] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2: stopping tablet replica
I20260924 16:10:52.174008 32099 tablet_replica.cc:330] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: stopping tablet replica
I20260924 16:10:52.174325 32097 raft_consensus.cc:2229] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:52.174338 32099 raft_consensus.cc:2229] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:52.174659 32099 raft_consensus.cc:2258] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:52.174661 32097 raft_consensus.cc:2258] T aad24b9127f74b2eb8917916918b8a4c P 760eedc493424f0491e1601ad74ad8e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:52.174796 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31657
I20260924 16:10:52.176283 32099 ts_tablet_manager.cc:1905] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:52.178447 32098 tablet_replica.cc:330] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: stopping tablet replica
I20260924 16:10:52.178838 32098 raft_consensus.cc:2229] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:52.179231 32099 ts_tablet_manager.cc:1918] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:52.179332 32099 log.cc:1196] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/aad24b9127f74b2eb8917916918b8a4c
I20260924 16:10:52.179361 32098 raft_consensus.cc:2258] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:52.179670 32099 ts_tablet_manager.cc:1939] T aad24b9127f74b2eb8917916918b8a4c P 7b794150ade34e7287bbe8f879d7a260: Deleting consensus metadata
I20260924 16:10:52.180891 31582 catalog_manager.cc:4841] TS 7b794150ade34e7287bbe8f879d7a260 (127.13.8.67:37033): tablet aad24b9127f74b2eb8917916918b8a4c (table test-table [id=736ed1d5205c4f27a1ea9798568e169f]) successfully deleted
I20260924 16:10:52.181247 32098 ts_tablet_manager.cc:1905] T aad24b9127f74b2eb8917916918b8a4c P 1753f0bb525d491a89da7e2fe0381ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 16:10:52.184206 31579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:43711 (error 108)
I20260924 16:10:52.184418 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31792
I20260924 16:10:52.185736 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.185409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:43711 (local address 127.0.0.1:34674)
0924 16:10:52.185553 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.185632 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260924 16:10:52.185899 31579 catalog_manager.cc:4584] TS 760eedc493424f0491e1601ad74ad8e2 (127.13.8.65:43711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aad24b9127f74b2eb8917916918b8a4c: Network error: Client connection negotiation failed: client connection to 127.13.8.65:43711: connect: Connection refused (error 111)
I20260924 16:10:52.191788 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31927
I20260924 16:10:52.192299 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.191950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39701 (local address 127.0.0.1:49198)
0924 16:10:52.192148 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.192203 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.66:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260924 16:10:52.197995 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 31562
2026-09-24T16:10:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2338 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T16:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:52Z Disabled control of system clock
I20260924 16:10:52.229738 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:45845
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:45845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:52.344193 32104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:52.344468 32104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:52.344506 32104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:52.348487 32104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:52.348544 32104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:52.348567 32104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:52.348583 32104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:52.348598 32104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:52.353861 32104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:45845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:45845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:52.355058 32104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:52.356458 32104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:52.365554 32109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.365617 32110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.365617 32112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:52.365994 32104 server_base.cc:1034] running on GCE node
I20260924 16:10:52.366649 32104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:52.367837 32104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:52.369046 32104 hybrid_clock.cc:648] HybridClock initialized: now 1790266252369024 us; error 31 us; skew 500 ppm
I20260924 16:10:52.371093 32104 webserver.cc:460] Webserver started at http://127.13.8.126:45789/ using document root <none> and password file <none>
I20260924 16:10:52.371697 32104 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:52.371788 32104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:52.372032 32104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:52.374066 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "f718a84e203e41bf818910c238bd56b1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "9df7fd9879fe824f21007f3f55ff9d75"
server_key_iv: "9dc6e9b7e722a29d2a33a16143f1f902"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.374696 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "f718a84e203e41bf818910c238bd56b1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "9df7fd9879fe824f21007f3f55ff9d75"
server_key_iv: "9dc6e9b7e722a29d2a33a16143f1f902"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.378358 32104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:10:52.380689 32119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.381915 32104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:52.382066 32104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "f718a84e203e41bf818910c238bd56b1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "9df7fd9879fe824f21007f3f55ff9d75"
server_key_iv: "9dc6e9b7e722a29d2a33a16143f1f902"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.382175 32104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:52.396729 32104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:52.397516 32104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:52.397716 32104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:52.404955 32104 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:45845
I20260924 16:10:52.404982 32171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:45845 every 8 connection(s)
I20260924 16:10:52.405937 32104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:52.409168 32172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:52.414871 32172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: Bootstrap starting.
I20260924 16:10:52.415549 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32104
I20260924 16:10:52.415716 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:52.416072 13345 external_mini_cluster.cc:1428] Setting key b7ddd7b253d4a8650b2a55157fd5b75f
I20260924 16:10:52.417774 32172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:52.418592 32172 log.cc:824] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:52.420470 32172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: No bootstrap required, opened a new log
I20260924 16:10:52.424131 32172 raft_consensus.cc:348] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } }
I20260924 16:10:52.424384 32172 raft_consensus.cc:374] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:52.424438 32172 raft_consensus.cc:729] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f718a84e203e41bf818910c238bd56b1, State: Initialized, Role: FOLLOWER
I20260924 16:10:52.424935 32172 consensus_queue.cc:260] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } }
I20260924 16:10:52.425099 32172 raft_consensus.cc:388] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:52.425179 32172 raft_consensus.cc:482] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:52.425304 32172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:52.425527 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.417220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.13.8.126:45845)
0924 16:10:52.417912 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.417937 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.417975 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:10:52.418662 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.418680 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.419122 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.419459 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:52.419875 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.420910 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.421950 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.422496 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.422676 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.423563 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:52.423568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:52.423582 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:52.423996 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:52.424048 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:52.424069 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:52.424176 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.424337 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.424653 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260924 16:10:52.426487 32172 raft_consensus.cc:504] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } }
I20260924 16:10:52.426846 32172 leader_election.cc:302] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f718a84e203e41bf818910c238bd56b1; no voters:
I20260924 16:10:52.427183 32172 leader_election.cc:288] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:52.427284 32177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:52.427525 32177 raft_consensus.cc:686] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [term 1 LEADER]: Becoming Leader. State: Replica: f718a84e203e41bf818910c238bd56b1, State: Running, Role: LEADER
I20260924 16:10:52.427848 32177 consensus_queue.cc:237] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } }
I20260924 16:10:52.428448 32172 sys_catalog.cc:564] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:52.429513 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f718a84e203e41bf818910c238bd56b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } } }
I20260924 16:10:52.429687 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:52.429947 32179 sys_catalog.cc:455] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f718a84e203e41bf818910c238bd56b1. Latest consensus state: current_term: 1 leader_uuid: "f718a84e203e41bf818910c238bd56b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f718a84e203e41bf818910c238bd56b1" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 45845 } } }
I20260924 16:10:52.430056 32179 sys_catalog.cc:458] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:52.430135 32186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:52.433387 32186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:52.438877 32186 catalog_manager.cc:1347] Generated new cluster ID: 7dafe8cc481d4968a205234995623806
I20260924 16:10:52.439002 32186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:52.447871 32186 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:52.448776 32186 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:52.452842 32186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: Generated new TSK 0
I20260924 16:10:52.453498 32186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:10:52.458981 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--builtin_ntp_servers=127.13.8.84:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:52.578665 32197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:52.578953 32197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:52.579032 32197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:52.583226 32197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:52.583300 32197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:52.583391 32197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:52.588364 32197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:52.589543 32197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:52.590983 32197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:52.600930 32202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.600934 32205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.600981 32203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:52.601410 32197 server_base.cc:1034] running on GCE node
I20260924 16:10:52.601817 32197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:52.602499 32197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:52.603662 32197 hybrid_clock.cc:648] HybridClock initialized: now 1790266252603643 us; error 29 us; skew 500 ppm
I20260924 16:10:52.605480 32197 webserver.cc:460] Webserver started at http://127.13.8.65:36497/ using document root <none> and password file <none>
I20260924 16:10:52.606253 32197 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:52.606328 32197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:52.606570 32197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:52.608615 32197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "76459bdb50e3cc894ebaf4c409254ce5"
server_key_iv: "bda227a6e7836214b99d569a0b313354"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.609177 32197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "76459bdb50e3cc894ebaf4c409254ce5"
server_key_iv: "bda227a6e7836214b99d569a0b313354"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.612789 32197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:10:52.615146 32211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.616351 32197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:52.616508 32197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "76459bdb50e3cc894ebaf4c409254ce5"
server_key_iv: "bda227a6e7836214b99d569a0b313354"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.616626 32197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:52.639094 32197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:52.639842 32197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:52.640043 32197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:52.640654 32197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:52.641641 32197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:52.641713 32197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.641789 32197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:52.641832 32197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.650906 32197 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:42067
I20260924 16:10:52.650975 32325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:42067 every 8 connection(s)
I20260924 16:10:52.651928 32197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:52.656761 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32197
I20260924 16:10:52.656903 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:52.657202 13345 external_mini_cluster.cc:1428] Setting key 5c6fb1f17ac9e6a36490deee230f66cf
I20260924 16:10:52.659224 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--builtin_ntp_servers=127.13.8.84:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:52.662362 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.653889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:37221 (local address 127.13.8.126:45845)
0924 16:10:52.654071 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.654075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.655189 (+ 1114us) server_negotiation.cc:407] Connection header received
0924 16:10:52.656662 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.656665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.656723 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.656827 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:52.658003 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.658697 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.660041 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.660404 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.660613 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.661802 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:52.661805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:52.661807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:52.661820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:52.661839 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:52.661847 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:52.661924 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.662118 (+ 194us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.662188 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 16:10:52.663530 32328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.654345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45845 (local address 127.13.8.65:37221)
0924 16:10:52.655036 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.655077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.656433 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.657035 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.657050 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.657140 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:52.657836 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.657855 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.658838 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.658842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.659870 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.659881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.660646 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.660649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.660804 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.661494 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:52.661495 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:52.661541 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:52.661617 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:52.661928 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:52.661941 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:52.661948 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.662116 (+ 168us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.662546 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":121,"threads_started":1}
I20260924 16:10:52.665215 32326 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45845
I20260924 16:10:52.665762 32326 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:52.666613 32326 heartbeater.cc:511] Master 127.13.8.126:45845 requested a full tablet report, sending...
I20260924 16:10:52.669251 32136 ts_manager.cc:194] Registered new tserver with Master: 3d59e7c6d0f147e58d973fe7f1c6a2c1 (127.13.8.65:42067)
I20260924 16:10:52.671509 32136 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.65:37221
W20260924 16:10:52.778640 32329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:52.778895 32329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:52.778934 32329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:52.782953 32329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:52.783002 32329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:52.783056 32329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:52.787842 32329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:52.788837 32329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:52.790259 32329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:52.800318 32334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.800325 32337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:52.800330 32335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:52.800565 32329 server_base.cc:1034] running on GCE node
I20260924 16:10:52.801018 32329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:52.801651 32329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:52.802843 32329 hybrid_clock.cc:648] HybridClock initialized: now 1790266252802827 us; error 25 us; skew 500 ppm
I20260924 16:10:52.804780 32329 webserver.cc:460] Webserver started at http://127.13.8.66:38565/ using document root <none> and password file <none>
I20260924 16:10:52.805434 32329 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:52.805526 32329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:52.805783 32329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:52.807915 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "305362798ae94e728fe3119ff562d3cc"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "edcdb1b759a9961686e72af6e04114fd"
server_key_iv: "07b11ccc6e3472edd0f853eeeb86b367"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.808516 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "305362798ae94e728fe3119ff562d3cc"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "edcdb1b759a9961686e72af6e04114fd"
server_key_iv: "07b11ccc6e3472edd0f853eeeb86b367"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.812232 32329 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260924 16:10:52.814697 32343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.815860 32329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:52.816033 32329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "305362798ae94e728fe3119ff562d3cc"
format_stamp: "Formatted at 2026-09-24 16:10:52 on dist-test-slave-vj6n"
server_key: "edcdb1b759a9961686e72af6e04114fd"
server_key_iv: "07b11ccc6e3472edd0f853eeeb86b367"
server_key_version: "encryptionkey@0"
I20260924 16:10:52.816179 32329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:52.836256 32329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:52.836961 32329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:52.837160 32329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:52.837755 32329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:52.838791 32329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:52.838871 32329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.839023 32329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:52.839071 32329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:52.848124 32329 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:39757
I20260924 16:10:52.848156 32457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:39757 every 8 connection(s)
I20260924 16:10:52.849035 32329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:52.856374 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32329
I20260924 16:10:52.856536 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:52.856907 13345 external_mini_cluster.cc:1428] Setting key c7e79b9d7383bc3caccd00dcca6b3ed7
I20260924 16:10:52.859097 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.851049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:51721 (local address 127.13.8.126:45845)
0924 16:10:52.851208 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:52.851212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:52.852403 (+ 1191us) server_negotiation.cc:407] Connection header received
0924 16:10:52.853759 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:52.853762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:52.853816 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:52.853908 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:52.854930 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.855490 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.856730 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.857072 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.857179 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.858538 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:52.858542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:52.858544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:52.858557 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:52.858581 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:52.858591 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:52.858671 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:52.858827 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 16:10:52.858898 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 16:10:52.859093 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--builtin_ntp_servers=127.13.8.84:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:52.859885 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:52.851530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45845 (local address 127.13.8.66:51721)
0924 16:10:52.852229 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:52.852269 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:52.853532 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:52.854074 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:52.854081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:52.854164 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:52.854763 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.854777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.855615 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.855617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.856583 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:52.856593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:52.857238 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:52.857241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:52.857453 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:52.858196 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:52.858199 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:52.858282 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:52.858369 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:52.858681 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:52.858688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:52.858693 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:10:52.858789 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 16:10:52.859011 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":134,"threads_started":1}
I20260924 16:10:52.861065 32458 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45845
I20260924 16:10:52.861331 32458 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:52.861956 32458 heartbeater.cc:511] Master 127.13.8.126:45845 requested a full tablet report, sending...
I20260924 16:10:52.863061 32136 ts_manager.cc:194] Registered new tserver with Master: 305362798ae94e728fe3119ff562d3cc (127.13.8.66:39757)
I20260924 16:10:52.863636 32136 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.66:51721
W20260924 16:10:52.989303 32461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:52.989727 32461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:52.989827 32461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:52.998836 32461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:52.998920 32461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:52.999049 32461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:53.005333 32461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:45845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:53.006521 32461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:53.007913 32461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:53.017772 32466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:53.017763 32469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:53.017763 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:53.017818 32461 server_base.cc:1034] running on GCE node
I20260924 16:10:53.018407 32461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:53.019111 32461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:53.020274 32461 hybrid_clock.cc:648] HybridClock initialized: now 1790266253020239 us; error 46 us; skew 500 ppm
I20260924 16:10:53.022132 32461 webserver.cc:460] Webserver started at http://127.13.8.67:43193/ using document root <none> and password file <none>
I20260924 16:10:53.022775 32461 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:53.022837 32461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:53.023074 32461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:53.025180 32461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "08f2764a21db45398eb56cda340f88eb"
format_stamp: "Formatted at 2026-09-24 16:10:53 on dist-test-slave-vj6n"
server_key: "cf97690d54a88b57032665834741c6a6"
server_key_iv: "25cbdcce0af7827425adc4e7eee2d126"
server_key_version: "encryptionkey@0"
I20260924 16:10:53.025741 32461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "08f2764a21db45398eb56cda340f88eb"
format_stamp: "Formatted at 2026-09-24 16:10:53 on dist-test-slave-vj6n"
server_key: "cf97690d54a88b57032665834741c6a6"
server_key_iv: "25cbdcce0af7827425adc4e7eee2d126"
server_key_version: "encryptionkey@0"
I20260924 16:10:53.029696 32461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:10:53.032284 32475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:53.033542 32461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:10:53.033666 32461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "08f2764a21db45398eb56cda340f88eb"
format_stamp: "Formatted at 2026-09-24 16:10:53 on dist-test-slave-vj6n"
server_key: "cf97690d54a88b57032665834741c6a6"
server_key_iv: "25cbdcce0af7827425adc4e7eee2d126"
server_key_version: "encryptionkey@0"
I20260924 16:10:53.033776 32461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:53.054308 32461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:53.055069 32461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:53.055282 32461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:53.055929 32461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:53.056921 32461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:53.056967 32461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:53.057039 32461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:53.057083 32461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:53.066376 32461 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:42701
I20260924 16:10:53.066478 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:42701 every 8 connection(s)
I20260924 16:10:53.067381 32461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:53.071677 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32461
I20260924 16:10:53.071805 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:10:53.072084 13345 external_mini_cluster.cc:1428] Setting key e5bd43277e82a17d290c4fa96d6bec8c
I20260924 16:10:53.077467 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.069597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:37451 (local address 127.13.8.126:45845)
0924 16:10:53.069767 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.069773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.070964 (+ 1191us) server_negotiation.cc:407] Connection header received
0924 16:10:53.072493 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.072496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.072563 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.072659 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:53.073868 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.074597 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.075718 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.075934 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.076009 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.077051 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:53.077054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:53.077056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:53.077078 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:53.077097 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:53.077105 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:53.077162 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.077303 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.077353 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 16:10:53.078464 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.070056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:45845 (local address 127.13.8.67:37451)
0924 16:10:53.070812 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.070853 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.072258 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.072854 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.072863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.072976 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:53.073727 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.073747 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.074711 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.074713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.075615 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.075624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.076068 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.076071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.076233 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.076800 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:53.076802 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:53.076844 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:53.076912 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:53.077178 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:53.077185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:53.077190 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.077287 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.077543 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":168,"threads_started":1}
I20260924 16:10:53.079623 32590 heartbeater.cc:347] Connected to a master server at 127.13.8.126:45845
I20260924 16:10:53.079861 32590 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:53.080391 32590 heartbeater.cc:511] Master 127.13.8.126:45845 requested a full tablet report, sending...
I20260924 16:10:53.081406 32136 ts_manager.cc:194] Registered new tserver with Master: 08f2764a21db45398eb56cda340f88eb (127.13.8.67:42701)
I20260924 16:10:53.081980 32136 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.67:37451
I20260924 16:10:53.086017 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:53.092064 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.088068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.13.8.126:45845)
0924 16:10:53.088220 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.088224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.088361 (+ 137us) server_negotiation.cc:407] Connection header received
0924 16:10:53.088486 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.088488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.088532 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.088617 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:53.088893 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.089414 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.090294 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.090549 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.090632 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.091503 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:53.091531 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:53.091533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:53.091545 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:53.091565 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:53.091574 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:53.091715 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.091912 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.091950 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 16:10:53.095218 32136 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:53.098443 32136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:53.113492 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.107662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.13.8.67:42701)
0924 16:10:53.108458 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.108471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.108498 (+ 27us) server_negotiation.cc:407] Connection header received
0924 16:10:53.108600 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.108607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.108822 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.109372 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:53.109522 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.110620 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.112139 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.112691 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.112817 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.113038 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.113226 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.113329 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":117,"threads_started":1}
I20260924 16:10:53.113487 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.107532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:42701 (local address 127.0.0.1:33114)
0924 16:10:53.108053 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.108102 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.108225 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.109022 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.109031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.109063 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:53.109212 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.109222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.110732 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.110737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.112013 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.112023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.112818 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.112822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.113012 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.113060 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.113194 (+ 134us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.113321 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":32,"thread_start_us":72,"threads_started":1}
I20260924 16:10:53.113752 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.107632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42067 (local address 127.0.0.1:43362)
0924 16:10:53.108749 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.108764 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.108846 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.109628 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.109631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.109648 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:53.109755 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.109762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.111294 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.111299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.112306 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.112316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.113385 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.113389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.113474 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.113498 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.113547 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.113614 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":47,"thread_start_us":108,"threads_started":1}
I20260924 16:10:53.114416 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.108355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.13.8.65:42067)
0924 16:10:53.109225 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.109239 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.109266 (+ 27us) server_negotiation.cc:407] Connection header received
0924 16:10:53.109346 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.109350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.109491 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.109656 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:53.109884 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.111129 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.112426 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.113244 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.113858 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.114064 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.114115 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.114254 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":161,"threads_started":1}
I20260924 16:10:53.115100 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.108320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:39757 (local address 127.0.0.1:37646)
0924 16:10:53.109107 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.109120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.109200 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.109783 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.109786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.109803 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:53.109887 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.109892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.111306 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.111310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.113258 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.113269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.114564 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.114568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.114811 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.114834 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.114905 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.114980 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":111,"thread_start_us":99,"threads_started":1}
I20260924 16:10:53.115624 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.108724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.13.8.66:39757)
0924 16:10:53.109197 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.109208 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.109233 (+ 25us) server_negotiation.cc:407] Connection header received
0924 16:10:53.109310 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.109315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.109503 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.109652 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:53.110031 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.111164 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.113390 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.114166 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.114301 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.114492 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.115342 (+ 850us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.115440 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":151,"threads_started":1}
I20260924 16:10:53.117210 32259 tablet_service.cc:1467] Processing CreateTablet for tablet 2999abe0050940d7a1f699485fa6136e (DEFAULT_TABLE table=test-table [id=737c1301195444abbdea72ad276da69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:53.117251 32523 tablet_service.cc:1467] Processing CreateTablet for tablet 2999abe0050940d7a1f699485fa6136e (DEFAULT_TABLE table=test-table [id=737c1301195444abbdea72ad276da69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:53.118191 32259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2999abe0050940d7a1f699485fa6136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:53.118358 32523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2999abe0050940d7a1f699485fa6136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:53.118417 32391 tablet_service.cc:1467] Processing CreateTablet for tablet 2999abe0050940d7a1f699485fa6136e (DEFAULT_TABLE table=test-table [id=737c1301195444abbdea72ad276da69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:53.119436 32391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2999abe0050940d7a1f699485fa6136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:53.125088 32607 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Bootstrap starting.
I20260924 16:10:53.126708 32608 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Bootstrap starting.
I20260924 16:10:53.127581 32607 tablet_bootstrap.cc:654] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:53.128512 32607 log.cc:824] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:53.128937 32608 tablet_bootstrap.cc:654] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:53.129266 32609 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Bootstrap starting.
I20260924 16:10:53.129779 32608 log.cc:824] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:53.130926 32607 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: No bootstrap required, opened a new log
I20260924 16:10:53.131151 32607 ts_tablet_manager.cc:1397] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260924 16:10:53.131477 32609 tablet_bootstrap.cc:654] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:53.132346 32609 log.cc:824] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:53.132750 32608 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: No bootstrap required, opened a new log
I20260924 16:10:53.132936 32608 ts_tablet_manager.cc:1397] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:53.134485 32609 tablet_bootstrap.cc:492] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: No bootstrap required, opened a new log
I20260924 16:10:53.134686 32609 ts_tablet_manager.cc:1397] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260924 16:10:53.135252 32607 raft_consensus.cc:348] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.135506 32607 raft_consensus.cc:374] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:53.135560 32607 raft_consensus.cc:729] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d59e7c6d0f147e58d973fe7f1c6a2c1, State: Initialized, Role: FOLLOWER
I20260924 16:10:53.136018 32607 consensus_queue.cc:260] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.137125 32607 ts_tablet_manager.cc:1428] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260924 16:10:53.137822 32326 heartbeater.cc:503] Master 127.13.8.126:45845 was elected leader, sending a full tablet report...
I20260924 16:10:53.137763 32608 raft_consensus.cc:348] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.138063 32608 raft_consensus.cc:374] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:53.138134 32608 raft_consensus.cc:729] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f2764a21db45398eb56cda340f88eb, State: Initialized, Role: FOLLOWER
I20260924 16:10:53.138639 32608 consensus_queue.cc:260] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.139117 32609 raft_consensus.cc:348] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.139372 32609 raft_consensus.cc:374] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:53.139426 32609 raft_consensus.cc:729] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305362798ae94e728fe3119ff562d3cc, State: Initialized, Role: FOLLOWER
I20260924 16:10:53.139686 32608 ts_tablet_manager.cc:1428] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260924 16:10:53.139863 32609 consensus_queue.cc:260] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.140515 32458 heartbeater.cc:503] Master 127.13.8.126:45845 was elected leader, sending a full tablet report...
I20260924 16:10:53.140635 32590 heartbeater.cc:503] Master 127.13.8.126:45845 was elected leader, sending a full tablet report...
I20260924 16:10:53.140831 32609 ts_tablet_manager.cc:1428] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
W20260924 16:10:53.153172 32327 tablet.cc:2369] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:10:53.183305 32615 raft_consensus.cc:482] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:53.183777 32615 raft_consensus.cc:504] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.185277 32615 leader_election.cc:288] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08f2764a21db45398eb56cda340f88eb (127.13.8.67:42701), 3d59e7c6d0f147e58d973fe7f1c6a2c1 (127.13.8.65:42067)
I20260924 16:10:53.190582 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.185627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:40893 (local address 127.13.8.67:42701)
0924 16:10:53.185770 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.185776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.186111 (+ 335us) server_negotiation.cc:407] Connection header received
0924 16:10:53.186177 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.186179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.186240 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.186334 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:53.186674 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.187465 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.189264 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.189863 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.189974 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.190042 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.190377 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.190453 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:10:53.190598 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.185875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:58711 (local address 127.13.8.65:42067)
0924 16:10:53.186031 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.186036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.186051 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:10:53.186112 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.186116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.186173 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.186337 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:53.186904 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.187666 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.189264 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.189863 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.189975 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.190038 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.190377 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.190452 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:10:53.190584 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.185441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:42701 (local address 127.13.8.66:40893)
0924 16:10:53.185849 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.185865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.185982 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.186349 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.186352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.186372 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:53.186500 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.186507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.187612 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.187616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.189118 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.189130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.190000 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.190003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.190168 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.190257 (+ 89us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.190345 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.190435 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":168}
I20260924 16:10:53.190593 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.185528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42067 (local address 127.13.8.66:58711)
0924 16:10:53.185927 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.185938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:53.186012 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:53.186645 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:53.186650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:53.186681 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:53.186782 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.186786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.187741 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.187744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.189120 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:53.189130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.190000 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.190003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:53.190194 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.190279 (+ 85us) client_negotiation.cc:769] Sending connection context
0924 16:10:53.190345 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 16:10:53.190435 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":168,"thread_start_us":124,"threads_started":1}
I20260924 16:10:53.191337 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2999abe0050940d7a1f699485fa6136e" candidate_uuid: "305362798ae94e728fe3119ff562d3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f2764a21db45398eb56cda340f88eb" is_pre_election: true
I20260924 16:10:53.191571 32280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2999abe0050940d7a1f699485fa6136e" candidate_uuid: "305362798ae94e728fe3119ff562d3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" is_pre_election: true
I20260924 16:10:53.191715 32543 raft_consensus.cc:2454] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 305362798ae94e728fe3119ff562d3cc in term 0.
I20260924 16:10:53.191883 32280 raft_consensus.cc:2454] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 305362798ae94e728fe3119ff562d3cc in term 0.
I20260924 16:10:53.192322 32344 leader_election.cc:302] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08f2764a21db45398eb56cda340f88eb, 305362798ae94e728fe3119ff562d3cc; no voters:
I20260924 16:10:53.192623 32615 raft_consensus.cc:2781] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:53.192698 32615 raft_consensus.cc:482] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:53.192729 32615 raft_consensus.cc:3037] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:53.193794 32615 raft_consensus.cc:504] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.194167 32615 leader_election.cc:288] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [CANDIDATE]: Term 1 election: Requested vote from peers 08f2764a21db45398eb56cda340f88eb (127.13.8.67:42701), 3d59e7c6d0f147e58d973fe7f1c6a2c1 (127.13.8.65:42067)
I20260924 16:10:53.194595 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2999abe0050940d7a1f699485fa6136e" candidate_uuid: "305362798ae94e728fe3119ff562d3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f2764a21db45398eb56cda340f88eb"
I20260924 16:10:53.194700 32280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2999abe0050940d7a1f699485fa6136e" candidate_uuid: "305362798ae94e728fe3119ff562d3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1"
I20260924 16:10:53.194702 32543 raft_consensus.cc:3037] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:53.194847 32280 raft_consensus.cc:3037] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:53.195875 32280 raft_consensus.cc:2454] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 305362798ae94e728fe3119ff562d3cc in term 1.
I20260924 16:10:53.195924 32543 raft_consensus.cc:2454] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 305362798ae94e728fe3119ff562d3cc in term 1.
I20260924 16:10:53.196247 32347 leader_election.cc:302] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 305362798ae94e728fe3119ff562d3cc, 3d59e7c6d0f147e58d973fe7f1c6a2c1; no voters:
I20260924 16:10:53.196470 32615 raft_consensus.cc:2781] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:53.196754 32615 raft_consensus.cc:686] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 1 LEADER]: Becoming Leader. State: Replica: 305362798ae94e728fe3119ff562d3cc, State: Running, Role: LEADER
I20260924 16:10:53.197191 32615 consensus_queue.cc:237] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } }
I20260924 16:10:53.200728 32135 catalog_manager.cc:5495] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 305362798ae94e728fe3119ff562d3cc (127.13.8.66). New cstate: current_term: 1 leader_uuid: "305362798ae94e728fe3119ff562d3cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "305362798ae94e728fe3119ff562d3cc" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 39757 } health_report { overall_health: HEALTHY } } }
I20260924 16:10:53.213727 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.210049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.13.8.66:39757)
0924 16:10:53.210188 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:53.210193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:53.210208 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:10:53.210379 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:53.210381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:53.210438 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:53.210550 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:53.210801 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.211293 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.212191 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:53.212407 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:53.212520 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:53.212837 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:53.213269 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:53.213438 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:53.213523 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 16:10:53.213559 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 16:10:53.219939 32280 raft_consensus.cc:1261] T 2999abe0050940d7a1f699485fa6136e P 3d59e7c6d0f147e58d973fe7f1c6a2c1 [term 1 FOLLOWER]: Refusing update from remote peer 305362798ae94e728fe3119ff562d3cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:53.220007 32543 raft_consensus.cc:1261] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 1 FOLLOWER]: Refusing update from remote peer 305362798ae94e728fe3119ff562d3cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:53.220781 32615 consensus_queue.cc:1035] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d59e7c6d0f147e58d973fe7f1c6a2c1" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 42067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:53.221042 32617 consensus_queue.cc:1035] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "08f2764a21db45398eb56cda340f88eb" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:53.239180 32136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:53.239392 32136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:53.242102 32136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f718a84e203e41bf818910c238bd56b1: Sending DeleteTablet for 3 replicas of tablet 2999abe0050940d7a1f699485fa6136e
I20260924 16:10:53.243053 32259 tablet_service.cc:1514] Processing DeleteTablet for tablet 2999abe0050940d7a1f699485fa6136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:53 UTC) from {username='slave'} at 127.0.0.1:43362
I20260924 16:10:53.243070 32523 tablet_service.cc:1514] Processing DeleteTablet for tablet 2999abe0050940d7a1f699485fa6136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:53 UTC) from {username='slave'} at 127.0.0.1:33114
I20260924 16:10:53.243145 32391 tablet_service.cc:1514] Processing DeleteTablet for tablet 2999abe0050940d7a1f699485fa6136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:53 UTC) from {username='slave'} at 127.0.0.1:37646
I20260924 16:10:53.243669 32630 tablet_replica.cc:330] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: stopping tablet replica
I20260924 16:10:53.243928 32631 tablet_replica.cc:330] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: stopping tablet replica
I20260924 16:10:53.244065 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32197
I20260924 16:10:53.245342 32630 raft_consensus.cc:2229] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:53.245879 32630 raft_consensus.cc:2258] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:53.246368 32631 raft_consensus.cc:2229] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:53.246713 32631 raft_consensus.cc:2258] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:53.248222 32631 ts_tablet_manager.cc:1905] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:53.248222 32630 ts_tablet_manager.cc:1905] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:53.251160 32631 ts_tablet_manager.cc:1918] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:53.251329 32631 log.cc:1196] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/2999abe0050940d7a1f699485fa6136e
I20260924 16:10:53.251727 32631 ts_tablet_manager.cc:1939] T 2999abe0050940d7a1f699485fa6136e P 08f2764a21db45398eb56cda340f88eb: Deleting consensus metadata
I20260924 16:10:53.251717 32630 ts_tablet_manager.cc:1918] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:53.251803 32630 log.cc:1196] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/2999abe0050940d7a1f699485fa6136e
I20260924 16:10:53.252121 32630 ts_tablet_manager.cc:1939] T 2999abe0050940d7a1f699485fa6136e P 305362798ae94e728fe3119ff562d3cc: Deleting consensus metadata
I20260924 16:10:53.253279 32120 catalog_manager.cc:4841] TS 08f2764a21db45398eb56cda340f88eb (127.13.8.67:42701): tablet 2999abe0050940d7a1f699485fa6136e (table test-table [id=737c1301195444abbdea72ad276da69b]) successfully deleted
I20260924 16:10:53.253279 32123 catalog_manager.cc:4841] TS 305362798ae94e728fe3119ff562d3cc (127.13.8.66:39757): tablet 2999abe0050940d7a1f699485fa6136e (table test-table [id=737c1301195444abbdea72ad276da69b]) successfully deleted
W20260924 16:10:53.254552 32123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:42067 (error 108)
I20260924 16:10:53.254614 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32329
I20260924 16:10:53.256148 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:53.255760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:42067 (local address 127.0.0.1:43374)
0924 16:10:53.255955 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:53.256042 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260924 16:10:53.256438 32123 catalog_manager.cc:4584] TS 3d59e7c6d0f147e58d973fe7f1c6a2c1 (127.13.8.65:42067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2999abe0050940d7a1f699485fa6136e: Network error: Client connection negotiation failed: client connection to 127.13.8.65:42067: connect: Connection refused (error 111)
I20260924 16:10:53.260950 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32461
I20260924 16:10:53.266980 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32104
2026-09-24T16:10:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1067 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:53 dist-test-slave-vj6n krb5kdc[32636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:53 dist-test-slave-vj6n krb5kdc[32636](info): set up 2 sockets
Sep 24 16:10:53 dist-test-slave-vj6n krb5kdc[32636](info): commencing operation
krb5kdc: starting...
W20260924 16:10:54.379292 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:10:54 dist-test-slave-vj6n krb5kdc[32636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:10:54.504540 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:38905
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:38905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:54.618534 32649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:54.618793 32649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:54.618870 32649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:54.622889 32649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:54.622949 32649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:54.622969 32649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:54.623008 32649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:54.623040 32649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:54.628345 32649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:38905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:38905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:54.629436 32649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:54.630585 32649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:54.639941 32655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:54.639957 32657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:54.639958 32654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:54.640165 32649 server_base.cc:1034] running on GCE node
I20260924 16:10:54.640853 32649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:54.642033 32649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:54.643211 32649 hybrid_clock.cc:648] HybridClock initialized: now 1790266254643181 us; error 46 us; skew 500 ppm
Sep 24 16:10:54 dist-test-slave-vj6n krb5kdc[32636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266254, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:54.645927 32649 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:10:54.646900 32649 webserver.cc:460] Webserver started at http://127.13.8.126:35259/ using document root <none> and password file <none>
I20260924 16:10:54.647522 32649 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:54.647621 32649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:54.647864 32649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:54.649964 32649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "f34305107215459399e0db8f1c949e91"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "3ae1ba85146f41ab8f130806d40b4b04"
server_key_iv: "7899dcc0590d3e5a1e812a8f6e7650ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.650605 32649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "f34305107215459399e0db8f1c949e91"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "3ae1ba85146f41ab8f130806d40b4b04"
server_key_iv: "7899dcc0590d3e5a1e812a8f6e7650ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.654433 32649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:54.656958 32664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:54.658363 32649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:54.658507 32649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "f34305107215459399e0db8f1c949e91"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "3ae1ba85146f41ab8f130806d40b4b04"
server_key_iv: "7899dcc0590d3e5a1e812a8f6e7650ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.658632 32649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:54.677457 32649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:54.680501 32649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:54.680709 32649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:54.687764 32649 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:38905
I20260924 16:10:54.687793 32717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:38905 every 8 connection(s)
I20260924 16:10:54.688755 32649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:54.690510 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32649
I20260924 16:10:54.690619 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:54.690934 13345 external_mini_cluster.cc:1428] Setting key 10cb90af3e456b81a539222cfe21612e
I20260924 16:10:54.691767 32718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:54.698611 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: Bootstrap starting.
Sep 24 16:10:54 dist-test-slave-vj6n krb5kdc[32636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:54.701128 32718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:54.701844 32718 log.cc:824] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:54.704211 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: No bootstrap required, opened a new log
I20260924 16:10:54.706379 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:54.692362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.13.8.126:38905)
0924 16:10:54.692790 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:54.692809 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:54.692852 (+ 43us) server_negotiation.cc:407] Connection header received
0924 16:10:54.693723 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:54.693746 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:54.694157 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:54.694549 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:54.695223 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.696717 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.697581 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.698272 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.698436 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:54.701026 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:54.701031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:54.701043 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:54.701071 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:54.703049 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:54.703467 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:54.703473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:54.703478 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:54.703655 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:54.703917 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:54.703920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:54.703922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:54.704411 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:54.704605 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:54.704867 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 16:10:54.705175 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260924 16:10:54.708029 32718 raft_consensus.cc:348] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } }
I20260924 16:10:54.708295 32718 raft_consensus.cc:374] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:54.708344 32718 raft_consensus.cc:729] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34305107215459399e0db8f1c949e91, State: Initialized, Role: FOLLOWER
I20260924 16:10:54.708786 32718 consensus_queue.cc:260] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } }
I20260924 16:10:54.708906 32718 raft_consensus.cc:388] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:54.708954 32718 raft_consensus.cc:482] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:54.709023 32718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:54.709810 32718 raft_consensus.cc:504] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } }
I20260924 16:10:54.710143 32718 leader_election.cc:302] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f34305107215459399e0db8f1c949e91; no voters:
I20260924 16:10:54.710487 32718 leader_election.cc:288] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:54.710603 32723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:54.710773 32723 raft_consensus.cc:686] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [term 1 LEADER]: Becoming Leader. State: Replica: f34305107215459399e0db8f1c949e91, State: Running, Role: LEADER
I20260924 16:10:54.711048 32723 consensus_queue.cc:237] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } }
I20260924 16:10:54.711756 32718 sys_catalog.cc:564] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:54.712728 32724 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f34305107215459399e0db8f1c949e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } } }
I20260924 16:10:54.712852 32724 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:54.713233 32732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:54.713213 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f34305107215459399e0db8f1c949e91. Latest consensus state: current_term: 1 leader_uuid: "f34305107215459399e0db8f1c949e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34305107215459399e0db8f1c949e91" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 38905 } } }
I20260924 16:10:54.713892 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:54.716313 32732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:54.721508 32732 catalog_manager.cc:1347] Generated new cluster ID: 3bc600429af74349b43e46ad801d741e
I20260924 16:10:54.721570 32732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:54.729075 32732 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:54.730387 32732 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:54.747805 32732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: Generated new TSK 0
I20260924 16:10:54.748662 32732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
Entry for principal kudu/127.13.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab.
I20260924 16:10:54.779301 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:54.892370 32744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:54.892635 32744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:54.892710 32744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:54.896780 32744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:54.896858 32744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:54.896946 32744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:54.902017 32744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.65.keytab
--principal=kudu/127.13.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:54.903118 32744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:54.904132 32744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:54.913616 32750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:54.913662 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:54.913630 32744 server_base.cc:1034] running on GCE node
W20260924 16:10:54.913652 32752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:54.914209 32744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:54.914965 32744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:54.916126 32744 hybrid_clock.cc:648] HybridClock initialized: now 1790266254916093 us; error 44 us; skew 500 ppm
Sep 24 16:10:54 dist-test-slave-vj6n krb5kdc[32636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266254, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:54.919117 32744 init.cc:377] Logged in from keytab as kudu/127.13.8.65@KRBTEST.COM (short username kudu)
I20260924 16:10:54.920078 32744 webserver.cc:460] Webserver started at http://127.13.8.65:43463/ using document root <none> and password file <none>
I20260924 16:10:54.920644 32744 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:54.920709 32744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:54.920892 32744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:54.923089 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "960ba5ddccfc4c4e91e15b001dab02b9"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "4fd4be4b7763fe576fb80d9ef61e1df5"
server_key_iv: "81a22217af89de085e2525b3f393571b"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.923645 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "960ba5ddccfc4c4e91e15b001dab02b9"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "4fd4be4b7763fe576fb80d9ef61e1df5"
server_key_iv: "81a22217af89de085e2525b3f393571b"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.927601 32744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:54.929950 32759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:54.931226 32744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:54.931329 32744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "960ba5ddccfc4c4e91e15b001dab02b9"
format_stamp: "Formatted at 2026-09-24 16:10:54 on dist-test-slave-vj6n"
server_key: "4fd4be4b7763fe576fb80d9ef61e1df5"
server_key_iv: "81a22217af89de085e2525b3f393571b"
server_key_version: "encryptionkey@0"
I20260924 16:10:54.931422 32744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:54.948158 32744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:54.951308 32744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:54.951467 32744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:54.952001 32744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:54.952798 32744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:54.952842 32744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:54.952883 32744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:54.952898 32744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:54.961786 32744 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:39729
I20260924 16:10:54.961864 405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:39729 every 8 connection(s)
I20260924 16:10:54.962815 32744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:54.965353 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 32744
I20260924 16:10:54.965488 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:54.965776 13345 external_mini_cluster.cc:1428] Setting key 65fe94615d49d47d459227b4dc3437df
Sep 24 16:10:54 dist-test-slave-vj6n krb5kdc[32636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266254, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.65@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.8.66@KRBTEST.COM; defaulting to no policy
I20260924 16:10:54.977538 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:54.964644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:41287 (local address 127.13.8.126:38905)
0924 16:10:54.964786 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:54.964790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:54.965655 (+ 865us) server_negotiation.cc:407] Connection header received
0924 16:10:54.967351 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:54.967354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:54.967406 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:54.967507 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:54.968689 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.969220 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.970746 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.970989 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.971159 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:54.973705 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:54.973709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:54.973711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:54.973734 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:54.975350 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:54.975910 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:54.975918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:54.975920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:54.975993 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:54.976365 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:54.976369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:54.976371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:54.976555 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:54.976665 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:54.977178 (+ 513us) server_negotiation.cc:299] Negotiation successful
0924 16:10:54.977321 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.66@KRBTEST.COM" created.
I20260924 16:10:54.978503 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:54.965019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38905 (local address 127.13.8.65:41287)
0924 16:10:54.965494 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:54.965546 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:54.967127 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:54.967695 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:54.967703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:54.968063 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:54.968540 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:54.968553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.969592 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.969595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:54.970618 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:54.970629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:54.971204 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:54.971208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:54.971385 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:54.972050 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:54.972069 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:54.973520 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:54.975473 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:54.975479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:54.975494 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:54.975798 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:54.976106 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:54.976109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:54.976111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:54.976262 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:54.976665 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:54.976672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:54.976930 (+ 258us) client_negotiation.cc:769] Sending connection context
0924 16:10:54.977124 (+ 194us) client_negotiation.cc:240] Negotiation successful
0924 16:10:54.977496 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":135,"threads_started":1}
I20260924 16:10:54.979894 406 heartbeater.cc:347] Connected to a master server at 127.13.8.126:38905
I20260924 16:10:54.980165 406 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:54.980693 406 heartbeater.cc:511] Master 127.13.8.126:38905 requested a full tablet report, sending...
I20260924 16:10:54.982316 32681 ts_manager.cc:194] Registered new tserver with Master: 960ba5ddccfc4c4e91e15b001dab02b9 (127.13.8.65:39729)
I20260924 16:10:54.983497 32681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.65@KRBTEST.COM'} at 127.13.8.65:41287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
Entry for principal kudu/127.13.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab.
I20260924 16:10:54.993572 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:55.106323 411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:55.106570 411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:55.106611 411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:55.110569 411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:55.110615 411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:55.110689 411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:55.116060 411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.66.keytab
--principal=kudu/127.13.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:55.117075 411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:55.118033 411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:55.127730 417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.127732 416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.127704 419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:55.127944 411 server_base.cc:1034] running on GCE node
I20260924 16:10:55.128352 411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:55.129014 411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:55.130270 411 hybrid_clock.cc:648] HybridClock initialized: now 1790266255130173 us; error 43 us; skew 500 ppm
Sep 24 16:10:55 dist-test-slave-vj6n krb5kdc[32636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266255, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:55.133307 411 init.cc:377] Logged in from keytab as kudu/127.13.8.66@KRBTEST.COM (short username kudu)
I20260924 16:10:55.134330 411 webserver.cc:460] Webserver started at http://127.13.8.66:33553/ using document root <none> and password file <none>
I20260924 16:10:55.134920 411 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:55.134992 411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:55.135265 411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:55.137292 411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "d9b160c488a448f3b9c468985fdcf140"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "afb689421cfe9acec38a201e00f43086"
server_key_iv: "0416ca165ec2bcd6208e356a0e3d2210"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.137917 411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "d9b160c488a448f3b9c468985fdcf140"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "afb689421cfe9acec38a201e00f43086"
server_key_iv: "0416ca165ec2bcd6208e356a0e3d2210"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.141821 411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260924 16:10:55.144464 426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.145704 411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:10:55.145843 411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "d9b160c488a448f3b9c468985fdcf140"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "afb689421cfe9acec38a201e00f43086"
server_key_iv: "0416ca165ec2bcd6208e356a0e3d2210"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.145947 411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:55.159811 411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:55.163144 411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:55.163348 411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:55.163938 411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:55.164750 411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:55.164798 411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.164868 411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:55.164911 411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.173640 411 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:37399
I20260924 16:10:55.173672 542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:37399 every 8 connection(s)
I20260924 16:10:55.174623 411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:55.180081 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 411
I20260924 16:10:55.180217 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:55.180527 13345 external_mini_cluster.cc:1428] Setting key 859ca36836d4b0e4e9a00a342ade1aac
Sep 24 16:10:55 dist-test-slave-vj6n krb5kdc[32636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266255, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.66@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:55.189254 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.176435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:57085 (local address 127.13.8.126:38905)
0924 16:10:55.176619 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.176622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.177466 (+ 844us) server_negotiation.cc:407] Connection header received
0924 16:10:55.178800 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.178803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.178849 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.178924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:55.180059 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.180878 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.182445 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.182867 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.182990 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.185673 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:55.185677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:55.185679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:55.185701 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:55.187212 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.187724 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.187732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.187733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.187805 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.188170 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.188175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.188177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.188356 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:55.188465 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.188951 (+ 486us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.189068 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 16:10:55.190037 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.176853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38905 (local address 127.13.8.66:57085)
0924 16:10:55.177308 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.177351 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.178602 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.179099 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.179106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.179441 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:55.179900 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.179913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.181034 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.181038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.182324 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.182334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.183018 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.183021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.183164 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.183836 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:55.183863 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:55.185459 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:55.187330 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:55.187335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:55.187350 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:55.187608 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:55.187905 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:55.187908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:55.187910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:55.188047 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:55.188465 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:55.188473 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:55.188721 (+ 248us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.188908 (+ 187us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.189231 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.13.8.67@KRBTEST.COM; defaulting to no policy
I20260924 16:10:55.191274 543 heartbeater.cc:347] Connected to a master server at 127.13.8.126:38905
I20260924 16:10:55.191533 543 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:55.192022 543 heartbeater.cc:511] Master 127.13.8.126:38905 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.67@KRBTEST.COM" created.
I20260924 16:10:55.193167 32681 ts_manager.cc:194] Registered new tserver with Master: d9b160c488a448f3b9c468985fdcf140 (127.13.8.66:37399)
I20260924 16:10:55.193775 32681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.66@KRBTEST.COM'} at 127.13.8.66:57085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
Entry for principal kudu/127.13.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab.
I20260924 16:10:55.209383 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:55.321617 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:55.321880 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:55.321923 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:55.326025 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:55.326081 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:55.326138 548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:55.331429 548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.67.keytab
--principal=kudu/127.13.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:55.332551 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:55.333595 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:55.343834 553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.343885 556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.343906 554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:55.344022 548 server_base.cc:1034] running on GCE node
I20260924 16:10:55.344496 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:55.345206 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:55.346372 548 hybrid_clock.cc:648] HybridClock initialized: now 1790266255346331 us; error 53 us; skew 500 ppm
Sep 24 16:10:55 dist-test-slave-vj6n krb5kdc[32636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266255, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:55.349365 548 init.cc:377] Logged in from keytab as kudu/127.13.8.67@KRBTEST.COM (short username kudu)
I20260924 16:10:55.350464 548 webserver.cc:460] Webserver started at http://127.13.8.67:37065/ using document root <none> and password file <none>
I20260924 16:10:55.351064 548 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:55.351156 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:55.351401 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:55.353494 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "41db5d05405648688339c50fcd0a28dc"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "a2c73406803fdf338ef6ab8a7eccab98"
server_key_iv: "bb307f47dd7451b3388ca5684747c967"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.354068 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "41db5d05405648688339c50fcd0a28dc"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "a2c73406803fdf338ef6ab8a7eccab98"
server_key_iv: "bb307f47dd7451b3388ca5684747c967"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.358121 548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260924 16:10:55.360808 563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.362154 548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 16:10:55.362380 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "41db5d05405648688339c50fcd0a28dc"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "a2c73406803fdf338ef6ab8a7eccab98"
server_key_iv: "bb307f47dd7451b3388ca5684747c967"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.362490 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:55.381749 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:55.384765 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:55.384984 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:55.385681 548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:55.386554 548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:55.386610 548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.386679 548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:55.386723 548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.395607 548 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:37725
I20260924 16:10:55.395609 679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:37725 every 8 connection(s)
I20260924 16:10:55.396581 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:55.405957 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 548
I20260924 16:10:55.406106 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
Sep 24 16:10:55 dist-test-slave-vj6n krb5kdc[32636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266255, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.67@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:10:55.406491 13345 external_mini_cluster.cc:1428] Setting key 88ed1e2caa15f519a4dc81a054e681b2
I20260924 16:10:55.410786 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.398454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:46929 (local address 127.13.8.126:38905)
0924 16:10:55.398616 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.398620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.399429 (+ 809us) server_negotiation.cc:407] Connection header received
0924 16:10:55.400789 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.400792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.400854 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.400963 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:55.402177 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.402772 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.404100 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.404330 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.404459 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.407112 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:55.407116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:55.407119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:55.407145 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:55.408928 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.409391 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.409393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.409395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.409467 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.409770 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.409773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.409774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.409939 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:55.410013 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.410571 (+ 558us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.410673 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:10:55.411724 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.398811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:38905 (local address 127.13.8.67:46929)
0924 16:10:55.399276 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.399319 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.400571 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.401157 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.401165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.401518 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:55.402026 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.402039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.402900 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.402904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.403968 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.403978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.404512 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.404516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.404693 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.405210 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:10:55.405226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:55.406961 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:55.409027 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:55.409032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:55.409044 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:55.409304 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:55.409559 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:10:55.409561 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:10:55.409563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:10:55.409691 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:10:55.410024 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:55.410030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:55.410328 (+ 298us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.410497 (+ 169us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.410764 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260924 16:10:55.412886 680 heartbeater.cc:347] Connected to a master server at 127.13.8.126:38905
I20260924 16:10:55.413125 680 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:55.413674 680 heartbeater.cc:511] Master 127.13.8.126:38905 requested a full tablet report, sending...
I20260924 16:10:55.414737 32681 ts_manager.cc:194] Registered new tserver with Master: 41db5d05405648688339c50fcd0a28dc (127.13.8.67:37725)
I20260924 16:10:55.415259 32681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.8.67@KRBTEST.COM'} at 127.13.8.67:46929
I20260924 16:10:55.420557 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:55.429888 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.422558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.13.8.126:38905)
0924 16:10:55.422679 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.422682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.422906 (+ 224us) server_negotiation.cc:407] Connection header received
0924 16:10:55.423010 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.423012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.423053 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.423128 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:55.423874 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.424366 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.425246 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.425448 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.425523 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.427015 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:55.427018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:55.427020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:55.427040 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:55.428383 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.428771 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.428773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.428774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.428844 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:55.429075 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:55.429078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:55.429079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:55.429275 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:55.429377 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.429678 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.429765 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:10:55.432667 32681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:55.435902 32681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:55.450325 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.444103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:37399 (local address 127.0.0.1:38016)
0924 16:10:55.445102 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.445115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.445196 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.445886 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.445889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.445909 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:55.446077 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.446084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.447586 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.447589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.448801 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.448812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.449912 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.449916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.450004 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.450027 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.450094 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.450166 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":33,"thread_start_us":90,"threads_started":1}
I20260924 16:10:55.450448 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.444828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.13.8.66:37399)
0924 16:10:55.445456 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.445469 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.445492 (+ 23us) server_negotiation.cc:407] Connection header received
0924 16:10:55.445574 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.445578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.445716 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.445888 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:55.446240 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.447431 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.448928 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.449743 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.449956 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.450115 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.450167 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.450285 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":102,"threads_started":1}
I20260924 16:10:55.450706 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.444145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.13.8.67:37725)
0924 16:10:55.445280 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.445293 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.445316 (+ 23us) server_negotiation.cc:407] Connection header received
0924 16:10:55.445409 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.445416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.445646 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.445777 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:55.446186 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.447160 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.448653 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.449532 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.449658 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.449819 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.450404 (+ 585us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.450524 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":93,"threads_started":1}
I20260924 16:10:55.450923 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.444039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:37725 (local address 127.0.0.1:37534)
0924 16:10:55.444364 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.444409 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.444524 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.445814 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.445822 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.445850 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:55.446066 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.446073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.447286 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.447292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.448514 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.448524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.449669 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.449673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.449788 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.449830 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.449941 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.450063 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":32,"thread_start_us":69,"threads_started":1}
I20260924 16:10:55.453229 475 tablet_service.cc:1467] Processing CreateTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 (DEFAULT_TABLE table=test-table [id=0086933593d44206817c0d1911271dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:55.453298 613 tablet_service.cc:1467] Processing CreateTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 (DEFAULT_TABLE table=test-table [id=0086933593d44206817c0d1911271dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:55.454107 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.444779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:39729 (local address 127.0.0.1:37356)
0924 16:10:55.445478 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.445491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.445571 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.446585 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.446589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.446609 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:55.446711 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.446717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.448103 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.448107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.449241 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.449249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.451246 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.451249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.453791 (+ 2542us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.453819 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.453904 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.453988 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":129,"threads_started":1}
I20260924 16:10:55.454243 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.445149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.13.8.65:39729)
0924 16:10:55.446109 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.446125 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.446150 (+ 25us) server_negotiation.cc:407] Connection header received
0924 16:10:55.446251 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.446256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.446432 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.446596 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:55.446836 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.447959 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.449351 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.450162 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.451017 (+ 855us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.451190 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.453959 (+ 2769us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.454065 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":61,"threads_started":1}
I20260924 16:10:55.454363 475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4847b56a9ad74f82a9cf49b1df696112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:55.454365 613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4847b56a9ad74f82a9cf49b1df696112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:55.457312 339 tablet_service.cc:1467] Processing CreateTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 (DEFAULT_TABLE table=test-table [id=0086933593d44206817c0d1911271dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:55.458425 339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4847b56a9ad74f82a9cf49b1df696112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:55.460749 697 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Bootstrap starting.
I20260924 16:10:55.462453 698 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Bootstrap starting.
I20260924 16:10:55.463615 697 tablet_bootstrap.cc:654] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:55.464406 697 log.cc:824] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:55.464699 698 tablet_bootstrap.cc:654] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:55.466096 698 log.cc:824] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:55.466338 700 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: Bootstrap starting.
I20260924 16:10:55.466732 697 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: No bootstrap required, opened a new log
I20260924 16:10:55.466948 697 ts_tablet_manager.cc:1397] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:55.468322 698 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: No bootstrap required, opened a new log
I20260924 16:10:55.468508 698 ts_tablet_manager.cc:1397] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260924 16:10:55.468611 700 tablet_bootstrap.cc:654] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:55.469894 700 log.cc:824] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:55.470134 697 raft_consensus.cc:348] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.470453 697 raft_consensus.cc:374] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:55.470525 697 raft_consensus.cc:729] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b160c488a448f3b9c468985fdcf140, State: Initialized, Role: FOLLOWER
I20260924 16:10:55.471006 697 consensus_queue.cc:260] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.471684 543 heartbeater.cc:503] Master 127.13.8.126:38905 was elected leader, sending a full tablet report...
I20260924 16:10:55.471925 700 tablet_bootstrap.cc:492] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: No bootstrap required, opened a new log
I20260924 16:10:55.472054 697 ts_tablet_manager.cc:1428] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260924 16:10:55.472088 700 ts_tablet_manager.cc:1397] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260924 16:10:55.472525 698 raft_consensus.cc:348] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.472797 698 raft_consensus.cc:374] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:55.472854 698 raft_consensus.cc:729] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41db5d05405648688339c50fcd0a28dc, State: Initialized, Role: FOLLOWER
I20260924 16:10:55.473145 703 raft_consensus.cc:482] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:55.473327 703 raft_consensus.cc:504] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.473331 698 consensus_queue.cc:260] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.474375 698 ts_tablet_manager.cc:1428] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260924 16:10:55.474727 680 heartbeater.cc:503] Master 127.13.8.126:38905 was elected leader, sending a full tablet report...
I20260924 16:10:55.475936 700 raft_consensus.cc:348] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.476202 700 raft_consensus.cc:374] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:55.476274 700 raft_consensus.cc:729] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960ba5ddccfc4c4e91e15b001dab02b9, State: Initialized, Role: FOLLOWER
I20260924 16:10:55.476706 700 consensus_queue.cc:260] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.477640 406 heartbeater.cc:503] Master 127.13.8.126:38905 was elected leader, sending a full tablet report...
I20260924 16:10:55.477691 700 ts_tablet_manager.cc:1428] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260924 16:10:55.477268 703 leader_election.cc:288] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41db5d05405648688339c50fcd0a28dc (127.13.8.67:37725), 960ba5ddccfc4c4e91e15b001dab02b9 (127.13.8.65:39729)
I20260924 16:10:55.479908 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.474861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:37725 (local address 127.13.8.66:53579)
0924 16:10:55.475158 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.475176 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.475263 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.475908 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.475912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.475930 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:55.476041 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.476048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.477101 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.477105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.478385 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.478399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.479419 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.479422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.479598 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.479633 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.479706 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.479777 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":24}
I20260924 16:10:55.480330 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.475446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53579 (local address 127.13.8.67:37725)
0924 16:10:55.475596 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.475601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.475614 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:10:55.475663 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.475665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.475715 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.475786 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:55.476196 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.476939 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.478539 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.479162 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.479254 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.479340 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.480158 (+ 818us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.480227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 16:10:55.480839 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4847b56a9ad74f82a9cf49b1df696112" candidate_uuid: "d9b160c488a448f3b9c468985fdcf140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41db5d05405648688339c50fcd0a28dc" is_pre_election: true
I20260924 16:10:55.481118 633 raft_consensus.cc:2454] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b160c488a448f3b9c468985fdcf140 in term 0.
I20260924 16:10:55.481658 427 leader_election.cc:302] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41db5d05405648688339c50fcd0a28dc, d9b160c488a448f3b9c468985fdcf140; no voters:
I20260924 16:10:55.481966 704 raft_consensus.cc:2781] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:55.482079 704 raft_consensus.cc:482] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:55.482137 704 raft_consensus.cc:3037] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:55.483009 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.476678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:39729 (local address 127.13.8.66:40647)
0924 16:10:55.478720 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.478734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:55.478852 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:55.479355 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:55.479360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:55.479377 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:55.479534 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.479542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.480580 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.480582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.481827 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:55.481837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.482700 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.482703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:55.482772 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.482792 (+ 20us) client_negotiation.cc:769] Sending connection context
0924 16:10:55.482854 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 16:10:55.482903 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1836,"mutex_wait_us":28,"thread_start_us":103,"threads_started":1}
I20260924 16:10:55.483078 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.477045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:40647 (local address 127.13.8.65:39729)
0924 16:10:55.477171 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.477178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.478795 (+ 1617us) server_negotiation.cc:407] Connection header received
0924 16:10:55.479096 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.479101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.479145 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.479199 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:55.479673 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.480440 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.481939 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.482571 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.482676 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.482743 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.482890 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.482957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:10:55.483568 704 raft_consensus.cc:504] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.483619 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4847b56a9ad74f82a9cf49b1df696112" candidate_uuid: "d9b160c488a448f3b9c468985fdcf140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" is_pre_election: true
I20260924 16:10:55.483861 359 raft_consensus.cc:2454] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b160c488a448f3b9c468985fdcf140 in term 0.
I20260924 16:10:55.483963 704 leader_election.cc:288] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [CANDIDATE]: Term 1 election: Requested vote from peers 41db5d05405648688339c50fcd0a28dc (127.13.8.67:37725), 960ba5ddccfc4c4e91e15b001dab02b9 (127.13.8.65:39729)
I20260924 16:10:55.484309 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4847b56a9ad74f82a9cf49b1df696112" candidate_uuid: "d9b160c488a448f3b9c468985fdcf140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "960ba5ddccfc4c4e91e15b001dab02b9"
I20260924 16:10:55.484429 359 raft_consensus.cc:3037] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:55.484545 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4847b56a9ad74f82a9cf49b1df696112" candidate_uuid: "d9b160c488a448f3b9c468985fdcf140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41db5d05405648688339c50fcd0a28dc"
I20260924 16:10:55.484663 633 raft_consensus.cc:3037] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:55.485668 359 raft_consensus.cc:2454] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b160c488a448f3b9c468985fdcf140 in term 1.
I20260924 16:10:55.485934 633 raft_consensus.cc:2454] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b160c488a448f3b9c468985fdcf140 in term 1.
I20260924 16:10:55.486014 428 leader_election.cc:302] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 960ba5ddccfc4c4e91e15b001dab02b9, d9b160c488a448f3b9c468985fdcf140; no voters:
I20260924 16:10:55.486321 704 raft_consensus.cc:2781] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:55.486449 704 raft_consensus.cc:686] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 1 LEADER]: Becoming Leader. State: Replica: d9b160c488a448f3b9c468985fdcf140, State: Running, Role: LEADER
I20260924 16:10:55.486865 704 consensus_queue.cc:237] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } }
I20260924 16:10:55.490166 32681 catalog_manager.cc:5495] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b160c488a448f3b9c468985fdcf140 (127.13.8.66). New cstate: current_term: 1 leader_uuid: "d9b160c488a448f3b9c468985fdcf140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9b160c488a448f3b9c468985fdcf140" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 37399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:55.501804 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.498325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.13.8.66:37399)
0924 16:10:55.498456 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.498459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.498472 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:10:55.498616 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.498618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.498670 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.498756 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:55.498963 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.499452 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.500246 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.500449 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.500523 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.500980 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:55.501375 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:55.501467 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.501572 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.501620 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:10:55.507927 359 raft_consensus.cc:1261] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 1 FOLLOWER]: Refusing update from remote peer d9b160c488a448f3b9c468985fdcf140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:55.507927 633 raft_consensus.cc:1261] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 1 FOLLOWER]: Refusing update from remote peer d9b160c488a448f3b9c468985fdcf140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:55.508667 703 consensus_queue.cc:1035] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "960ba5ddccfc4c4e91e15b001dab02b9" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 39729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:55.508910 704 consensus_queue.cc:1035] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41db5d05405648688339c50fcd0a28dc" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 37725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:55.517833 711 mvcc.cc:205] Tried to move back new op lower bound from 7332930582552629248 to 7332930582477299712. Current Snapshot: MvccSnapshot[applied={T|T < 7332930582552629248}]
I20260924 16:10:55.526396 32680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:55.526582 32680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:55.528985 32680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f34305107215459399e0db8f1c949e91: Sending DeleteTablet for 3 replicas of tablet 4847b56a9ad74f82a9cf49b1df696112
I20260924 16:10:55.529839 613 tablet_service.cc:1514] Processing DeleteTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:55 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:37534
I20260924 16:10:55.529840 475 tablet_service.cc:1514] Processing DeleteTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:55 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:38016
I20260924 16:10:55.530429 720 tablet_replica.cc:330] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: stopping tablet replica
I20260924 16:10:55.530457 339 tablet_service.cc:1514] Processing DeleteTablet for tablet 4847b56a9ad74f82a9cf49b1df696112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:55 UTC) from {username='kudu', principal='kudu/127.13.8.126@KRBTEST.COM'} at 127.0.0.1:37356
I20260924 16:10:55.530804 720 raft_consensus.cc:2229] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:55.531034 721 tablet_replica.cc:330] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9: stopping tablet replica
I20260924 16:10:55.531313 720 raft_consensus.cc:2258] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:55.531458 719 tablet_replica.cc:330] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: stopping tablet replica
I20260924 16:10:55.531499 721 raft_consensus.cc:2229] T 4847b56a9ad74f82a9cf49b1df696112 P 960ba5ddccfc4c4e91e15b001dab02b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:55.531783 719 raft_consensus.cc:2229] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:55.531875 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32744
I20260924 16:10:55.533340 720 ts_tablet_manager.cc:1905] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:55.534135 719 raft_consensus.cc:2258] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:55.536052 719 ts_tablet_manager.cc:1905] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:55.536206 720 ts_tablet_manager.cc:1918] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:55.536311 720 log.cc:1196] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-1/wal/wals/4847b56a9ad74f82a9cf49b1df696112
I20260924 16:10:55.536669 720 ts_tablet_manager.cc:1939] T 4847b56a9ad74f82a9cf49b1df696112 P d9b160c488a448f3b9c468985fdcf140: Deleting consensus metadata
I20260924 16:10:55.538107 32668 catalog_manager.cc:4841] TS d9b160c488a448f3b9c468985fdcf140 (127.13.8.66:37399): tablet 4847b56a9ad74f82a9cf49b1df696112 (table test-table [id=0086933593d44206817c0d1911271dfe]) successfully deleted
I20260924 16:10:55.539160 719 ts_tablet_manager.cc:1918] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:55.539292 719 log.cc:1196] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/4847b56a9ad74f82a9cf49b1df696112
I20260924 16:10:55.539683 719 ts_tablet_manager.cc:1939] T 4847b56a9ad74f82a9cf49b1df696112 P 41db5d05405648688339c50fcd0a28dc: Deleting consensus metadata
I20260924 16:10:55.540902 32665 catalog_manager.cc:4841] TS 41db5d05405648688339c50fcd0a28dc (127.13.8.67:37725): tablet 4847b56a9ad74f82a9cf49b1df696112 (table test-table [id=0086933593d44206817c0d1911271dfe]) successfully deleted
W20260924 16:10:55.542018 32666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:39729 (error 108)
I20260924 16:10:55.542299 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 411
I20260924 16:10:55.543363 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.543002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:39729 (local address 127.0.0.1:37358)
0924 16:10:55.543175 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:55.543256 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:39729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260924 16:10:55.543654 32666 catalog_manager.cc:4584] TS 960ba5ddccfc4c4e91e15b001dab02b9 (127.13.8.65:39729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4847b56a9ad74f82a9cf49b1df696112: Network error: Client connection negotiation failed: client connection to 127.13.8.65:39729: connect: Connection refused (error 111)
I20260924 16:10:55.549391 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 548
I20260924 16:10:55.555737 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 32649
2026-09-24T16:10:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2290 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T16:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:55Z Disabled control of system clock
I20260924 16:10:55.587196 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36149
--webserver_interface=127.13.8.126
--webserver_port=0
--builtin_ntp_servers=127.13.8.84:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:36149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:55.699098 726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:55.699386 726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:55.699422 726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:55.703583 726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:55.703650 726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:55.703665 726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:55.703683 726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:55.703698 726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:55.708763 726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:36149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:55.709954 726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:55.711396 726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:55.720188 734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.720206 732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:55.720350 726 server_base.cc:1034] running on GCE node
W20260924 16:10:55.720216 731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:55.721053 726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:55.722162 726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:55.723377 726 hybrid_clock.cc:648] HybridClock initialized: now 1790266255723347 us; error 50 us; skew 500 ppm
I20260924 16:10:55.725203 726 webserver.cc:460] Webserver started at http://127.13.8.126:33029/ using document root <none> and password file <none>
I20260924 16:10:55.725792 726 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:55.725888 726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:55.726132 726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:55.728199 726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "bb6b34d25b764f28ad34ba11770df6ef"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "d43fcdec0403fd8449362bd87ca33aba"
server_key_iv: "7754f5ad84256e570d19a9e877247b3f"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.728760 726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "bb6b34d25b764f28ad34ba11770df6ef"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "d43fcdec0403fd8449362bd87ca33aba"
server_key_iv: "7754f5ad84256e570d19a9e877247b3f"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.732393 726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:10:55.734764 741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:55.735946 726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:55.736094 726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "bb6b34d25b764f28ad34ba11770df6ef"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "d43fcdec0403fd8449362bd87ca33aba"
server_key_iv: "7754f5ad84256e570d19a9e877247b3f"
server_key_version: "encryptionkey@0"
I20260924 16:10:55.736225 726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:55.765771 726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:55.766618 726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:55.766836 726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:55.774081 726 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:36149
I20260924 16:10:55.774093 794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:36149 every 8 connection(s)
I20260924 16:10:55.775110 726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:55.778455 795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:55.783391 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 726
I20260924 16:10:55.783531 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:55.783844 13345 external_mini_cluster.cc:1428] Setting key fe15e7c62e29d7ae631c01f256891090
I20260924 16:10:55.784586 795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: Bootstrap starting.
I20260924 16:10:55.787135 795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:55.787916 795 log.cc:824] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:55.790000 795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: No bootstrap required, opened a new log
I20260924 16:10:55.794328 795 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } }
I20260924 16:10:55.794660 795 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:55.794724 795 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6b34d25b764f28ad34ba11770df6ef, State: Initialized, Role: FOLLOWER
I20260924 16:10:55.795222 795 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } }
I20260924 16:10:55.795375 795 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:55.795441 795 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:55.795558 795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:55.796484 795 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } }
I20260924 16:10:55.796849 795 leader_election.cc:302] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb6b34d25b764f28ad34ba11770df6ef; no voters:
I20260924 16:10:55.797160 795 leader_election.cc:288] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:55.797264 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:55.785744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.13.8.126:36149)
0924 16:10:55.786775 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:55.786798 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:55.786848 (+ 50us) server_negotiation.cc:407] Connection header received
0924 16:10:55.787899 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:55.787926 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:55.788507 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:55.788924 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:55.789488 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.791285 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.792575 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:55.793400 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:55.793590 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:55.794605 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:55.794612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:55.794631 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:55.795244 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:55.795324 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:55.795355 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:55.795499 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:55.795648 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 16:10:55.796114 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":150,"threads_started":1}
I20260924 16:10:55.797286 801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:55.797766 801 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [term 1 LEADER]: Becoming Leader. State: Replica: bb6b34d25b764f28ad34ba11770df6ef, State: Running, Role: LEADER
I20260924 16:10:55.798118 801 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } }
I20260924 16:10:55.798586 795 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:55.799937 803 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb6b34d25b764f28ad34ba11770df6ef. Latest consensus state: current_term: 1 leader_uuid: "bb6b34d25b764f28ad34ba11770df6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } } }
I20260924 16:10:55.800073 803 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:55.800457 802 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb6b34d25b764f28ad34ba11770df6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6b34d25b764f28ad34ba11770df6ef" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36149 } } }
I20260924 16:10:55.800560 802 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef [sys.catalog]: This master's current role is: LEADER
W20260924 16:10:55.804234 816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:10:55.804381 816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:10:55.806293 811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:55.807397 811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:55.814132 811 catalog_manager.cc:1347] Generated new cluster ID: 096b5272649a405189b7cf5b48326c17
I20260924 16:10:55.814199 811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:55.831539 811 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:55.832815 811 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:55.838279 811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: Generated new TSK 0
I20260924 16:10:55.838959 811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:10:55.843278 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.65:0
--local_ip_for_outbound_sockets=127.13.8.65
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--builtin_ntp_servers=127.13.8.84:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 16:10:55.973955 820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:55.974246 820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:55.974344 820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:55.978497 820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:55.978554 820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:55.978636 820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.65
I20260924 16:10:55.983580 820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.8.65
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.65
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:55.984714 820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:55.986129 820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:55.996176 829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.996176 825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:55.996213 827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:55.996488 820 server_base.cc:1034] running on GCE node
I20260924 16:10:55.996986 820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:55.997622 820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:55.998839 820 hybrid_clock.cc:648] HybridClock initialized: now 1790266255998814 us; error 28 us; skew 500 ppm
I20260924 16:10:56.000833 820 webserver.cc:460] Webserver started at http://127.13.8.65:34269/ using document root <none> and password file <none>
I20260924 16:10:56.001449 820 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:56.001514 820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:56.001789 820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:56.003994 820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data/instance:
uuid: "ce24aed8ed124953901a508eecd10537"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "10f8b19fcbcbaed36b8b2b585916983b"
server_key_iv: "8e561ee232a8a05bf4fed718f20d9327"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.004611 820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance:
uuid: "ce24aed8ed124953901a508eecd10537"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "10f8b19fcbcbaed36b8b2b585916983b"
server_key_iv: "8e561ee232a8a05bf4fed718f20d9327"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.008455 820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260924 16:10:56.011016 835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.012212 820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:10:56.012337 820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal
uuid: "ce24aed8ed124953901a508eecd10537"
format_stamp: "Formatted at 2026-09-24 16:10:55 on dist-test-slave-vj6n"
server_key: "10f8b19fcbcbaed36b8b2b585916983b"
server_key_iv: "8e561ee232a8a05bf4fed718f20d9327"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.012444 820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:56.024257 820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:56.024950 820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:56.025138 820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:56.025710 820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:56.026697 820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:56.026743 820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.026805 820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:56.026837 820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.035833 820 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.65:33949
I20260924 16:10:56.035853 950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.65:33949 every 8 connection(s)
I20260924 16:10:56.036813 820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/data/info.pb
I20260924 16:10:56.040421 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 820
I20260924 16:10:56.040546 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-0/wal/instance
I20260924 16:10:56.040911 13345 external_mini_cluster.cc:1428] Setting key 3ad29bb5e1e184f941a10172733cb211
I20260924 16:10:56.043089 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.66:0
--local_ip_for_outbound_sockets=127.13.8.66
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--builtin_ntp_servers=127.13.8.84:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:56.047508 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.038780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36539 (local address 127.13.8.126:36149)
0924 16:10:56.038920 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.038924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.040093 (+ 1169us) server_negotiation.cc:407] Connection header received
0924 16:10:56.041953 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.041956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.042012 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.042122 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:56.043426 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.043965 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.045253 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.045534 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.045619 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.047009 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:56.047012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:56.047014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:56.047034 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:56.047053 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:56.047061 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:56.047121 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.047329 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.047395 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:10:56.048934 953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.039257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36149 (local address 127.13.8.65:36539)
0924 16:10:56.039946 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.039987 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.041707 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.042355 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.042362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.042485 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:56.043222 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.043248 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.044114 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.044120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.045127 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.045137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.045714 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.045721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.045964 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.046682 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:56.046687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:56.046748 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:56.046842 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:56.047156 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:56.047170 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:56.047178 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.047330 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.047690 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260924 16:10:56.050414 951 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36149
I20260924 16:10:56.050707 951 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:56.051321 951 heartbeater.cc:511] Master 127.13.8.126:36149 requested a full tablet report, sending...
I20260924 16:10:56.053051 759 ts_manager.cc:194] Registered new tserver with Master: ce24aed8ed124953901a508eecd10537 (127.13.8.65:33949)
I20260924 16:10:56.054395 759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.65:36539
W20260924 16:10:56.158033 954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:56.158340 954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:56.158417 954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:56.162377 954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:56.162457 954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:56.162551 954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.66
I20260924 16:10:56.167201 954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.8.66
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.66
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:56.168295 954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:56.169693 954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:56.179289 962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:56.179307 960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:56.179297 959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:56.179699 954 server_base.cc:1034] running on GCE node
I20260924 16:10:56.180035 954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:56.180641 954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:56.181800 954 hybrid_clock.cc:648] HybridClock initialized: now 1790266256181786 us; error 25 us; skew 500 ppm
I20260924 16:10:56.183626 954 webserver.cc:460] Webserver started at http://127.13.8.66:37625/ using document root <none> and password file <none>
I20260924 16:10:56.184176 954 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:56.184235 954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:56.184403 954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:56.186474 954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data/instance:
uuid: "2be774783c6e47cfa276557084c5beea"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "167bc086807655771b5584a3f874ea1d"
server_key_iv: "71860d35e85cf191d74525e92f253779"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.186981 954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance:
uuid: "2be774783c6e47cfa276557084c5beea"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "167bc086807655771b5584a3f874ea1d"
server_key_iv: "71860d35e85cf191d74525e92f253779"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.190537 954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:10:56.192929 968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.194121 954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:10:56.194267 954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal
uuid: "2be774783c6e47cfa276557084c5beea"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "167bc086807655771b5584a3f874ea1d"
server_key_iv: "71860d35e85cf191d74525e92f253779"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.194370 954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:56.224248 954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:56.224926 954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:56.225073 954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:56.225631 954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:56.226550 954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:56.226596 954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.226636 954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:56.226660 954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.235577 954 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.66:44229
I20260924 16:10:56.235611 1082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.66:44229 every 8 connection(s)
I20260924 16:10:56.236481 954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/data/info.pb
I20260924 16:10:56.239758 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 954
I20260924 16:10:56.239920 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-1/wal/instance
I20260924 16:10:56.240345 13345 external_mini_cluster.cc:1428] Setting key 3c51eaacaa5c7f5d317fae89d25ec037
I20260924 16:10:56.242767 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.8.67:0
--local_ip_for_outbound_sockets=127.13.8.67
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--builtin_ntp_servers=127.13.8.84:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 16:10:56.249542 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.238310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.66:53989 (local address 127.13.8.126:36149)
0924 16:10:56.238444 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.238448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.240876 (+ 2428us) server_negotiation.cc:407] Connection header received
0924 16:10:56.243149 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.243155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.243222 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.243314 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:56.244748 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.245582 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.247320 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.247555 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.247667 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.248995 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:56.249000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:56.249002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:56.249016 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:56.249041 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:56.249053 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:56.249116 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.249341 (+ 225us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.249401 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:10:56.250607 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.238674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36149 (local address 127.13.8.66:53989)
0924 16:10:56.239395 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.239451 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.242822 (+ 3371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.243537 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.243550 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.243674 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:56.244510 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.244539 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.245755 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.245758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.247159 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.247171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.247719 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.247723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.247934 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.248681 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:56.248683 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:56.248736 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:56.248818 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:56.249186 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:56.249195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:56.249200 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.249340 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.249792 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":110,"threads_started":1}
I20260924 16:10:56.251777 1083 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36149
I20260924 16:10:56.252069 1083 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:56.252807 1083 heartbeater.cc:511] Master 127.13.8.126:36149 requested a full tablet report, sending...
I20260924 16:10:56.253985 759 ts_manager.cc:194] Registered new tserver with Master: 2be774783c6e47cfa276557084c5beea (127.13.8.66:44229)
I20260924 16:10:56.254688 759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.66:53989
W20260924 16:10:56.366519 1086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:56.366835 1086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:56.366920 1086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:56.371004 1086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:56.371083 1086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:56.371176 1086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.8.67
I20260924 16:10:56.376248 1086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.8.67
--webserver_port=0
--tserver_master_addrs=127.13.8.126:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.8.67
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:56.377331 1086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:56.378834 1086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:56.389235 1094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:56.389161 1091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:56.389181 1092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:56.389757 1086 server_base.cc:1034] running on GCE node
I20260924 16:10:56.390118 1086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:56.390751 1086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:56.391920 1086 hybrid_clock.cc:648] HybridClock initialized: now 1790266256391900 us; error 33 us; skew 500 ppm
I20260924 16:10:56.393925 1086 webserver.cc:460] Webserver started at http://127.13.8.67:42367/ using document root <none> and password file <none>
I20260924 16:10:56.394651 1086 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:56.394721 1086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:56.394942 1086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:56.397127 1086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data/instance:
uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "422b66ce03a45c6f747ec717816894f7"
server_key_iv: "74a7c1a84c248731c4785c8d1a74e1ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.397718 1086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance:
uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "422b66ce03a45c6f747ec717816894f7"
server_key_iv: "74a7c1a84c248731c4785c8d1a74e1ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.401736 1086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260924 16:10:56.404309 1100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.405803 1086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:10:56.405920 1086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal
uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85"
format_stamp: "Formatted at 2026-09-24 16:10:56 on dist-test-slave-vj6n"
server_key: "422b66ce03a45c6f747ec717816894f7"
server_key_iv: "74a7c1a84c248731c4785c8d1a74e1ba"
server_key_version: "encryptionkey@0"
I20260924 16:10:56.406040 1086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:56.424777 1086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:56.425455 1086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:56.425652 1086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:56.426292 1086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 16:10:56.427220 1086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 16:10:56.427268 1086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.427336 1086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 16:10:56.427376 1086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:56.436625 1086 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.67:44005
I20260924 16:10:56.436661 1215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.67:44005 every 8 connection(s)
I20260924 16:10:56.437484 1086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/data/info.pb
I20260924 16:10:56.440090 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1086
I20260924 16:10:56.440225 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal/instance
I20260924 16:10:56.440583 13345 external_mini_cluster.cc:1428] Setting key 68014ce4298e76455e54ed3dab42bedd
I20260924 16:10:56.447269 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.439311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.67:39745 (local address 127.13.8.126:36149)
0924 16:10:56.439453 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.439456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.440847 (+ 1391us) server_negotiation.cc:407] Connection header received
0924 16:10:56.442572 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.442575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.442620 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.442710 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:56.443624 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.444170 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.445321 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.445566 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.445657 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.446786 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:56.446789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:56.446791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:56.446803 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:56.446822 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:56.446830 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:56.446893 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.447110 (+ 217us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.447162 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:10:56.448123 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.439772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:36149 (local address 127.13.8.67:39745)
0924 16:10:56.440681 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.440734 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.442346 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.442893 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.442904 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.443020 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:10:56.443493 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.443506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.444295 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.444298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.445191 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.445201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.445712 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.445715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.445855 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.446550 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 16:10:56.446552 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:10:56.446591 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 16:10:56.446651 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:10:56.446938 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:10:56.446947 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:10:56.446952 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.447076 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.447326 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":165,"threads_started":1}
I20260924 16:10:56.449237 1216 heartbeater.cc:347] Connected to a master server at 127.13.8.126:36149
I20260924 16:10:56.449481 1216 heartbeater.cc:464] Registering TS with master...
I20260924 16:10:56.450007 1216 heartbeater.cc:511] Master 127.13.8.126:36149 requested a full tablet report, sending...
I20260924 16:10:56.451085 759 ts_manager.cc:194] Registered new tserver with Master: c3f6f8eb42374dfb885cc0fc8ccc8d85 (127.13.8.67:44005)
I20260924 16:10:56.451674 759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.8.67:39745
I20260924 16:10:56.454828 13345 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 16:10:56.460295 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.456750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.13.8.126:36149)
0924 16:10:56.456866 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.456869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.457021 (+ 152us) server_negotiation.cc:407] Connection header received
0924 16:10:56.457115 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.457117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.457154 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.457220 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:56.457544 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.458051 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.458950 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.459158 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.459237 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.459949 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:56.459952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:56.459954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:56.459965 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:56.459983 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:56.459991 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:56.460051 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.460146 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.460194 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 16:10:56.462903 759 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 16:10:56.465858 759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 16:10:56.480568 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.474483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:44229 (local address 127.0.0.1:60222)
0924 16:10:56.475200 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.475250 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.475384 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.476222 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.476230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.476257 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:56.476417 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.476422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.477617 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.477623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.478943 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.478955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.479984 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.479988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.480098 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.480141 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.480266 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.480401 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":57,"thread_start_us":68,"threads_started":1}
I20260924 16:10:56.480572 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.474615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.13.8.66:44229)
0924 16:10:56.475034 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.475047 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.475336 (+ 289us) server_negotiation.cc:407] Connection header received
0924 16:10:56.475618 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.475627 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.475842 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.475979 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:56.476532 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.477503 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.479058 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.479860 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.479971 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.480165 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.480294 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.480388 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":180,"threads_started":1}
I20260924 16:10:56.480815 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.474730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:33949 (local address 127.0.0.1:36444)
0924 16:10:56.475482 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.475496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.475591 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.476696 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.476699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.476718 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:56.476803 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.476809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.478154 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.478158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.479521 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.479530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.480454 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.480457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.480561 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.480591 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.480642 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.480711 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":159,"thread_start_us":95,"threads_started":1}
I20260924 16:10:56.481444 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.475743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.13.8.65:33949)
0924 16:10:56.476276 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.476287 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.476309 (+ 22us) server_negotiation.cc:407] Connection header received
0924 16:10:56.476378 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.476383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.476557 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.476706 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:56.476904 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.478036 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.479641 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.480325 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.480973 (+ 648us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.481162 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.481215 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.481297 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":96,"threads_started":1}
I20260924 16:10:56.481443 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.475208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:44005 (local address 127.0.0.1:35412)
0924 16:10:56.476163 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.476177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.476271 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.476904 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.476907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.476921 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:56.476984 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.476988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.478493 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.478496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.479774 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.479786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.481055 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.481059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.481157 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.481181 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.481257 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.481323 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":43,"thread_start_us":93,"threads_started":1}
I20260924 16:10:56.482077 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.475406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.13.8.67:44005)
0924 16:10:56.475856 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.475869 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.476290 (+ 421us) server_negotiation.cc:407] Connection header received
0924 16:10:56.476378 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.476383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.476557 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.476707 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:56.477093 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.478366 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.479921 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.480694 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.480803 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.480994 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.481306 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.481883 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":147,"threads_started":1}
I20260924 16:10:56.483733 883 tablet_service.cc:1467] Processing CreateTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f (DEFAULT_TABLE table=test-table [id=b50555b9e34c455f8af4e153359a23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:56.483696 1016 tablet_service.cc:1467] Processing CreateTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f (DEFAULT_TABLE table=test-table [id=b50555b9e34c455f8af4e153359a23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:56.484604 883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:56.484604 1016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:56.488337 1149 tablet_service.cc:1467] Processing CreateTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f (DEFAULT_TABLE table=test-table [id=b50555b9e34c455f8af4e153359a23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 16:10:56.489399 1149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:56.491035 1233 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Bootstrap starting.
I20260924 16:10:56.492398 1234 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Bootstrap starting.
I20260924 16:10:56.493446 1233 tablet_bootstrap.cc:654] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:56.494359 1233 log.cc:824] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:56.494563 1234 tablet_bootstrap.cc:654] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:56.495348 1234 log.cc:824] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:56.496587 1236 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Bootstrap starting.
I20260924 16:10:56.497020 1233 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: No bootstrap required, opened a new log
I20260924 16:10:56.497218 1233 ts_tablet_manager.cc:1397] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260924 16:10:56.497296 1234 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: No bootstrap required, opened a new log
I20260924 16:10:56.497467 1234 ts_tablet_manager.cc:1397] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 16:10:56.498615 1236 tablet_bootstrap.cc:654] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:56.499486 1236 log.cc:824] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:56.501156 1236 tablet_bootstrap.cc:492] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: No bootstrap required, opened a new log
I20260924 16:10:56.501346 1236 ts_tablet_manager.cc:1397] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260924 16:10:56.501718 1234 raft_consensus.cc:348] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.501718 1233 raft_consensus.cc:348] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.501987 1233 raft_consensus.cc:374] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:56.501987 1234 raft_consensus.cc:374] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:56.502048 1234 raft_consensus.cc:729] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce24aed8ed124953901a508eecd10537, State: Initialized, Role: FOLLOWER
I20260924 16:10:56.502048 1233 raft_consensus.cc:729] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be774783c6e47cfa276557084c5beea, State: Initialized, Role: FOLLOWER
I20260924 16:10:56.502537 1233 consensus_queue.cc:260] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.502530 1234 consensus_queue.cc:260] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.503538 1233 ts_tablet_manager.cc:1428] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260924 16:10:56.503528 1234 ts_tablet_manager.cc:1428] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260924 16:10:56.503899 951 heartbeater.cc:503] Master 127.13.8.126:36149 was elected leader, sending a full tablet report...
I20260924 16:10:56.504094 1083 heartbeater.cc:503] Master 127.13.8.126:36149 was elected leader, sending a full tablet report...
I20260924 16:10:56.505630 1236 raft_consensus.cc:348] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.505916 1236 raft_consensus.cc:374] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:56.505975 1236 raft_consensus.cc:729] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f6f8eb42374dfb885cc0fc8ccc8d85, State: Initialized, Role: FOLLOWER
I20260924 16:10:56.506459 1236 consensus_queue.cc:260] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.507109 1216 heartbeater.cc:503] Master 127.13.8.126:36149 was elected leader, sending a full tablet report...
I20260924 16:10:56.507527 1236 ts_tablet_manager.cc:1428] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260924 16:10:56.538064 952 tablet.cc:2369] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 16:10:56.563066 1239 raft_consensus.cc:482] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 16:10:56.563334 1239 raft_consensus.cc:504] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.564548 1239 leader_election.cc:288] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2be774783c6e47cfa276557084c5beea (127.13.8.66:44229), c3f6f8eb42374dfb885cc0fc8ccc8d85 (127.13.8.67:44005)
I20260924 16:10:56.570251 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.564943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:36759 (local address 127.13.8.67:44005)
0924 16:10:56.565083 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.565087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.565239 (+ 152us) server_negotiation.cc:407] Connection header received
0924 16:10:56.565326 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.565329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.565386 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.565471 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:56.565920 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.566727 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.568861 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.569467 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.569580 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.569642 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.570017 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.570096 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:10:56.570259 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.564836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.8.65:46073 (local address 127.13.8.66:44229)
0924 16:10:56.564970 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.564975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.565182 (+ 207us) server_negotiation.cc:407] Connection header received
0924 16:10:56.565327 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.565330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.565388 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.565471 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 16:10:56.565920 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.566751 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.568861 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.569467 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.569600 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.569649 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.570026 (+ 377us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.570107 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 16:10:56.570327 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.564754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:44229 (local address 127.13.8.65:46073)
0924 16:10:56.565074 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.565090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.565213 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.565503 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.565506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.565547 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:56.565797 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.565802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.566854 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.566858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.568706 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.568719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.569710 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.569714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.569878 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.569912 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.569992 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.570109 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":302,"spinlock_wait_cycles":30464,"thread_start_us":74,"threads_started":1}
I20260924 16:10:56.570696 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.564800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.67:44005 (local address 127.13.8.65:36759)
0924 16:10:56.565151 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.565163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 16:10:56.565235 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:10:56.565503 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:10:56.565506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:10:56.565549 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 16:10:56.565789 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.565798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.566854 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.566858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.568706 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:10:56.568719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.569710 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.569714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:10:56.569870 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.569922 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 16:10:56.569992 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 16:10:56.570114 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":514,"spinlock_wait_cycles":7296,"thread_start_us":92,"threads_started":1}
I20260924 16:10:56.570991 1036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5955f9bb842a4ed98e0f04cd70dc2d6f" candidate_uuid: "ce24aed8ed124953901a508eecd10537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2be774783c6e47cfa276557084c5beea" is_pre_election: true
I20260924 16:10:56.571342 1036 raft_consensus.cc:2454] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce24aed8ed124953901a508eecd10537 in term 0.
I20260924 16:10:56.571396 1169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5955f9bb842a4ed98e0f04cd70dc2d6f" candidate_uuid: "ce24aed8ed124953901a508eecd10537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" is_pre_election: true
I20260924 16:10:56.571667 1169 raft_consensus.cc:2454] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce24aed8ed124953901a508eecd10537 in term 0.
I20260924 16:10:56.571882 837 leader_election.cc:302] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2be774783c6e47cfa276557084c5beea, ce24aed8ed124953901a508eecd10537; no voters:
I20260924 16:10:56.572197 1239 raft_consensus.cc:2781] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 16:10:56.572285 1239 raft_consensus.cc:482] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 16:10:56.572316 1239 raft_consensus.cc:3037] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:56.573360 1239 raft_consensus.cc:504] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.573745 1239 leader_election.cc:288] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [CANDIDATE]: Term 1 election: Requested vote from peers 2be774783c6e47cfa276557084c5beea (127.13.8.66:44229), c3f6f8eb42374dfb885cc0fc8ccc8d85 (127.13.8.67:44005)
I20260924 16:10:56.574092 1036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5955f9bb842a4ed98e0f04cd70dc2d6f" candidate_uuid: "ce24aed8ed124953901a508eecd10537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2be774783c6e47cfa276557084c5beea"
I20260924 16:10:56.574188 1036 raft_consensus.cc:3037] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:56.575219 1036 raft_consensus.cc:2454] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce24aed8ed124953901a508eecd10537 in term 1.
I20260924 16:10:56.575440 1169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5955f9bb842a4ed98e0f04cd70dc2d6f" candidate_uuid: "ce24aed8ed124953901a508eecd10537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85"
I20260924 16:10:56.575484 837 leader_election.cc:302] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2be774783c6e47cfa276557084c5beea, ce24aed8ed124953901a508eecd10537; no voters:
I20260924 16:10:56.575543 1169 raft_consensus.cc:3037] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:56.575711 1239 raft_consensus.cc:2781] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:56.575948 1239 raft_consensus.cc:686] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 1 LEADER]: Becoming Leader. State: Replica: ce24aed8ed124953901a508eecd10537, State: Running, Role: LEADER
I20260924 16:10:56.576325 1239 consensus_queue.cc:237] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } }
I20260924 16:10:56.576880 1169 raft_consensus.cc:2454] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce24aed8ed124953901a508eecd10537 in term 1.
I20260924 16:10:56.579552 759 catalog_manager.cc:5495] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce24aed8ed124953901a508eecd10537 (127.13.8.65). New cstate: current_term: 1 leader_uuid: "ce24aed8ed124953901a508eecd10537" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce24aed8ed124953901a508eecd10537" member_type: VOTER last_known_addr { host: "127.13.8.65" port: 33949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 } health_report { overall_health: UNKNOWN } } }
I20260924 16:10:56.601650 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.598151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36448 (local address 127.13.8.65:33949)
0924 16:10:56.598360 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:56.598364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:56.598377 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:10:56.598483 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:56.598485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:56.598541 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:56.598653 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:10:56.598913 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.599505 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.600302 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:56.600556 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:56.600639 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:56.600903 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:10:56.601315 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:10:56.601403 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:56.601458 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 16:10:56.601503 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260924 16:10:56.607874 1036 raft_consensus.cc:1261] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 1 FOLLOWER]: Refusing update from remote peer ce24aed8ed124953901a508eecd10537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:56.607873 1169 raft_consensus.cc:1261] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 1 FOLLOWER]: Refusing update from remote peer ce24aed8ed124953901a508eecd10537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:10:56.608597 1239 consensus_queue.cc:1035] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2be774783c6e47cfa276557084c5beea" member_type: VOTER last_known_addr { host: "127.13.8.66" port: 44229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:56.608800 1244 consensus_queue.cc:1035] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f6f8eb42374dfb885cc0fc8ccc8d85" member_type: VOTER last_known_addr { host: "127.13.8.67" port: 44005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:10:56.619001 1248 mvcc.cc:205] Tried to move back new op lower bound from 7332930587056697344 to 7332930586939039744. Current Snapshot: MvccSnapshot[applied={T|T < 7332930587056697344}]
I20260924 16:10:56.620615 1247 mvcc.cc:205] Tried to move back new op lower bound from 7332930587056697344 to 7332930586939039744. Current Snapshot: MvccSnapshot[applied={T|T < 7332930587056697344 or (T in {7332930587056697344})}]
I20260924 16:10:56.625168 759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:56.625305 759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 16:10:56.627889 759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb6b34d25b764f28ad34ba11770df6ef: Sending DeleteTablet for 3 replicas of tablet 5955f9bb842a4ed98e0f04cd70dc2d6f
I20260924 16:10:56.628665 1016 tablet_service.cc:1514] Processing DeleteTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:56 UTC) from {username='slave'} at 127.0.0.1:60222
I20260924 16:10:56.628680 883 tablet_service.cc:1514] Processing DeleteTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:56 UTC) from {username='slave'} at 127.0.0.1:36444
I20260924 16:10:56.628777 1149 tablet_service.cc:1514] Processing DeleteTablet for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 16:10:56 UTC) from {username='slave'} at 127.0.0.1:35412
I20260924 16:10:56.629293 1257 tablet_replica.cc:330] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537: stopping tablet replica
I20260924 16:10:56.629315 1259 tablet_replica.cc:330] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: stopping tablet replica
I20260924 16:10:56.629424 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 820
I20260924 16:10:56.629670 1259 raft_consensus.cc:2229] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:56.629787 1257 raft_consensus.cc:2229] T 5955f9bb842a4ed98e0f04cd70dc2d6f P ce24aed8ed124953901a508eecd10537 [term 1 LEADER]: Raft consensus shutting down.
I20260924 16:10:56.630157 1259 raft_consensus.cc:2258] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:56.631760 1259 ts_tablet_manager.cc:1905] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:56.634398 1258 tablet_replica.cc:330] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: stopping tablet replica
I20260924 16:10:56.634718 1258 raft_consensus.cc:2229] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 16:10:56.635097 1258 raft_consensus.cc:2258] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 16:10:56.635243 1259 ts_tablet_manager.cc:1918] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 16:10:56.635350 1259 log.cc:1196] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Deleting WAL directory at /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790266001279336-13345-0/minicluster-data/ts-2/wal/wals/5955f9bb842a4ed98e0f04cd70dc2d6f
I20260924 16:10:56.635658 1259 ts_tablet_manager.cc:1939] T 5955f9bb842a4ed98e0f04cd70dc2d6f P c3f6f8eb42374dfb885cc0fc8ccc8d85: Deleting consensus metadata
I20260924 16:10:56.636610 1258 ts_tablet_manager.cc:1905] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 16:10:56.637073 744 catalog_manager.cc:4841] TS c3f6f8eb42374dfb885cc0fc8ccc8d85 (127.13.8.67:44005): tablet 5955f9bb842a4ed98e0f04cd70dc2d6f (table test-table [id=b50555b9e34c455f8af4e153359a23bf]) successfully deleted
I20260924 16:10:56.639004 1258 ts_tablet_manager.cc:1918] T 5955f9bb842a4ed98e0f04cd70dc2d6f P 2be774783c6e47cfa276557084c5beea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260924 16:10:56.639058 744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.8.65:33949 (error 108)
I20260924 16:10:56.639245 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 954
I20260924 16:10:56.640479 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.640110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.65:33949 (local address 127.0.0.1:36452)
0924 16:10:56.640287 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.640373 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.65:33949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260924 16:10:56.640666 744 catalog_manager.cc:4584] TS ce24aed8ed124953901a508eecd10537 (127.13.8.65:33949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5955f9bb842a4ed98e0f04cd70dc2d6f: Network error: Client connection negotiation failed: client connection to 127.13.8.65:33949: connect: Connection refused (error 111)
I20260924 16:10:56.645906 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1086
I20260924 16:10:56.646955 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:56.646080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.66:44229 (local address 127.0.0.1:60226)
0924 16:10:56.646786 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0924 16:10:56.646849 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.66:44229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604}
I20260924 16:10:56.651680 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 726
2026-09-24T16:10:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1094 ms)
[----------] 6 tests from EncryptionPolicyTest (10704 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:56 dist-test-slave-vj6n krb5kdc[1263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:56 dist-test-slave-vj6n krb5kdc[1263](info): set up 2 sockets
Sep 24 16:10:56 dist-test-slave-vj6n krb5kdc[1263](info): commencing operation
krb5kdc: starting...
W20260924 16:10:57.695298 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:10:57 dist-test-slave-vj6n krb5kdc[1263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:10:57.815054 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:57.930537 1277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:57.930840 1277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:57.930882 1277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:57.934958 1277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:57.935029 1277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:57.935050 1277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:57.935066 1277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:57.935081 1277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:57.940523 1277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:57.941672 1277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:57.942827 1277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:57.951819 1285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:57.951814 1282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:57.951872 1277 server_base.cc:1034] running on GCE node
W20260924 16:10:57.951814 1283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:57.952764 1277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:57.953974 1277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:57.955163 1277 hybrid_clock.cc:648] HybridClock initialized: now 1790266257955159 us; error 64 us; skew 500 ppm
Sep 24 16:10:57 dist-test-slave-vj6n krb5kdc[1263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266257, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:57.957898 1277 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:10:57.958925 1277 webserver.cc:460] Webserver started at http://127.0.0.1:43965/ using document root <none> and password file <none>
I20260924 16:10:57.959524 1277 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:57.959623 1277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:57.959900 1277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:57.962062 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "23990645976f4a33a3dd466ad306450b"
format_stamp: "Formatted at 2026-09-24 16:10:57 on dist-test-slave-vj6n"
server_key: "432436de9153f1dc53c33350a413fd3b"
server_key_iv: "d14dfed4d42affa22ce94b11f07776b8"
server_key_version: "encryptionkey@0"
I20260924 16:10:57.962707 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "23990645976f4a33a3dd466ad306450b"
format_stamp: "Formatted at 2026-09-24 16:10:57 on dist-test-slave-vj6n"
server_key: "432436de9153f1dc53c33350a413fd3b"
server_key_iv: "d14dfed4d42affa22ce94b11f07776b8"
server_key_version: "encryptionkey@0"
I20260924 16:10:57.966580 1277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:10:57.969331 1293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:57.970927 1277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:10:57.971074 1277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "23990645976f4a33a3dd466ad306450b"
format_stamp: "Formatted at 2026-09-24 16:10:57 on dist-test-slave-vj6n"
server_key: "432436de9153f1dc53c33350a413fd3b"
server_key_iv: "d14dfed4d42affa22ce94b11f07776b8"
server_key_version: "encryptionkey@0"
I20260924 16:10:57.971202 1277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:57.984194 1277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:57.987389 1277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:57.987610 1277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:57.995187 1277 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41675
I20260924 16:10:57.995211 1346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41675 every 8 connection(s)
I20260924 16:10:57.996260 1277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:57.999183 1347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:58.001657 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1277
I20260924 16:10:58.001811 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:58.002110 13345 external_mini_cluster.cc:1428] Setting key 690e1cf4bb79dbf679e9197a8e39d711
I20260924 16:10:58.005750 1347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b: Bootstrap starting.
I20260924 16:10:58.008112 1347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:58.008881 1347 log.cc:824] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:58.011229 1347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b: No bootstrap required, opened a new log
Sep 24 16:10:58 dist-test-slave-vj6n krb5kdc[1263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 16:10:58.014442 1347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260924 16:10:58.014737 1347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:58.014798 1347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23990645976f4a33a3dd466ad306450b, State: Initialized, Role: FOLLOWER
I20260924 16:10:58.015296 1347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260924 16:10:58.015470 1347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:58.015554 1347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:58.015652 1347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:58.016769 1347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260924 16:10:58.017117 1347 leader_election.cc:302] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23990645976f4a33a3dd466ad306450b; no voters:
I20260924 16:10:58.017462 1347 leader_election.cc:288] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:58.017584 1352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:58.017721 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:58.003705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40663 (local address 127.0.0.1:41675)
0924 16:10:58.004186 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:58.004201 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:58.004257 (+ 56us) server_negotiation.cc:407] Connection header received
0924 16:10:58.004906 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:58.004923 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:58.005288 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:58.005648 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:58.006540 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:58.007544 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:58.008497 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:58.008979 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:58.009157 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:58.012079 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:58.012084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:58.012099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:58.012127 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:58.014462 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:58.015053 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:58.015061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:58.015067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:58.015201 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:58.015556 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:58.015562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:58.015564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:58.015971 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:58.016152 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:58.016489 (+ 337us) server_negotiation.cc:299] Negotiation successful
0924 16:10:58.016741 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":141,"threads_started":1}
I20260924 16:10:58.017788 1352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [term 1 LEADER]: Becoming Leader. State: Replica: 23990645976f4a33a3dd466ad306450b, State: Running, Role: LEADER
I20260924 16:10:58.018086 1352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260924 16:10:58.018774 1347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:58.019796 1353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23990645976f4a33a3dd466ad306450b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260924 16:10:58.019919 1353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:58.020167 1353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23990645976f4a33a3dd466ad306450b. Latest consensus state: current_term: 1 leader_uuid: "23990645976f4a33a3dd466ad306450b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23990645976f4a33a3dd466ad306450b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260924 16:10:58.020237 1353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:58.020495 1361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:58.021396 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:10:58.024283 1361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:58.030465 1361 catalog_manager.cc:1347] Generated new cluster ID: 3315587910994a39b2ff3ce5c017418d
I20260924 16:10:58.030534 1361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:58.034546 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:58.023485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60215 (local address 127.0.0.1:41675)
0924 16:10:58.023588 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:58.023593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:58.026138 (+ 2545us) server_negotiation.cc:407] Connection header received
0924 16:10:58.026337 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:58.026340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:58.026394 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:58.026480 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:58.027163 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:58.027733 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:58.028742 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:58.029080 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:58.029184 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:58.030962 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:58.030966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:58.030969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:10:58.030996 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:58.032649 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:58.033210 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:58.033213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:58.033215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:58.033301 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:10:58.033641 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:10:58.033644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:10:58.033646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:10:58.033833 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:58.033967 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:58.034291 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 16:10:58.034385 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 16:10:58.049021 1361 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:58.050401 1361 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:58.063356 1361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23990645976f4a33a3dd466ad306450b: Generated new TSK 0
I20260924 16:10:58.064288 1361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:10:58.070395 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1277
W20260924 16:10:58.076565 1261 connection.cc:535] client connection to 127.0.0.1:41675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41675): Connection reset by peer (error 104)
2026-09-24T16:10:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:58 dist-test-slave-vj6n krb5kdc[1379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:58 dist-test-slave-vj6n krb5kdc[1379](info): set up 2 sockets
Sep 24 16:10:58 dist-test-slave-vj6n krb5kdc[1379](info): commencing operation
krb5kdc: starting...
W20260924 16:10:59.123453 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:10:59 dist-test-slave-vj6n krb5kdc[1379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:10:59.247025 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:10:59.357556 1392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:10:59.357833 1392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:10:59.357913 1392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:10:59.361928 1392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:10:59.361997 1392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:10:59.362017 1392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:10:59.362072 1392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:10:59.362104 1392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:10:59.367445 1392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34779
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:10:59.368580 1392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:10:59.369647 1392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:10:59.378230 1397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:59.378356 1392 server_base.cc:1034] running on GCE node
W20260924 16:10:59.378330 1400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:10:59.378263 1398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:10:59.379119 1392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:10:59.380211 1392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:10:59.381373 1392 hybrid_clock.cc:648] HybridClock initialized: now 1790266259381343 us; error 53 us; skew 500 ppm
Sep 24 16:10:59 dist-test-slave-vj6n krb5kdc[1379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266259, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:10:59.384163 1392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:10:59.385105 1392 webserver.cc:460] Webserver started at http://127.0.0.1:41551/ using document root <none> and password file <none>
I20260924 16:10:59.385689 1392 fs_manager.cc:359] Metadata directory not provided
I20260924 16:10:59.385749 1392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:10:59.386000 1392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:10:59.388248 1392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "267d700c78264b30abf8a97244cee9a4"
format_stamp: "Formatted at 2026-09-24 16:10:59 on dist-test-slave-vj6n"
server_key: "ea0e28616e7f2db7b65d4df2e46e3324"
server_key_iv: "f03fa05a3414b6b720f3c4440737adaa"
server_key_version: "encryptionkey@0"
I20260924 16:10:59.388839 1392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "267d700c78264b30abf8a97244cee9a4"
format_stamp: "Formatted at 2026-09-24 16:10:59 on dist-test-slave-vj6n"
server_key: "ea0e28616e7f2db7b65d4df2e46e3324"
server_key_iv: "f03fa05a3414b6b720f3c4440737adaa"
server_key_version: "encryptionkey@0"
I20260924 16:10:59.392886 1392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260924 16:10:59.395689 1407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:10:59.396950 1392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:10:59.397099 1392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "267d700c78264b30abf8a97244cee9a4"
format_stamp: "Formatted at 2026-09-24 16:10:59 on dist-test-slave-vj6n"
server_key: "ea0e28616e7f2db7b65d4df2e46e3324"
server_key_iv: "f03fa05a3414b6b720f3c4440737adaa"
server_key_version: "encryptionkey@0"
I20260924 16:10:59.397204 1392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:10:59.416885 1392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:10:59.419979 1392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:10:59.420190 1392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:10:59.427464 1392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34779
I20260924 16:10:59.427521 1461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34779 every 8 connection(s)
I20260924 16:10:59.428701 1392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:10:59.431650 1462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:10:59.432435 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1392
I20260924 16:10:59.432576 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:10:59.432900 13345 external_mini_cluster.cc:1428] Setting key c024024b4455079d9c7767d8ce44190e
I20260924 16:10:59.438009 1462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: Bootstrap starting.
I20260924 16:10:59.440387 1462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: Neither blocks nor log segments found. Creating new log.
I20260924 16:10:59.441130 1462 log.cc:824] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: Log is configured to *not* fsync() on all Append() calls
I20260924 16:10:59.442991 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:59.434423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55175 (local address 127.0.0.1:34779)
0924 16:10:59.434888 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:59.434904 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:59.434943 (+ 39us) server_negotiation.cc:407] Connection header received
0924 16:10:59.435654 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:59.435671 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:59.436020 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:59.436394 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:59.437012 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:59.438663 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:59.439585 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:59.440087 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:59.440273 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:59.441144 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:59.441150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:59.441173 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:59.441614 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:59.441669 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:59.441694 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:59.441836 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:59.442022 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 16:10:59.442169 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":127,"threads_started":1}
I20260924 16:10:59.443209 1462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: No bootstrap required, opened a new log
I20260924 16:10:59.445853 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:10:59.446975 1462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } }
I20260924 16:10:59.447288 1462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:10:59.447366 1462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267d700c78264b30abf8a97244cee9a4, State: Initialized, Role: FOLLOWER
I20260924 16:10:59.447772 1462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } }
I20260924 16:10:59.447948 1462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:10:59.448025 1462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:10:59.448122 1462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:10:59.448994 1462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } }
I20260924 16:10:59.449322 1462 leader_election.cc:302] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267d700c78264b30abf8a97244cee9a4; no voters:
I20260924 16:10:59.449652 1462 leader_election.cc:288] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:10:59.449746 1472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:10:59.450021 1472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 267d700c78264b30abf8a97244cee9a4, State: Running, Role: LEADER
I20260924 16:10:59.450457 1472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } }
I20260924 16:10:59.450976 1462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:10:59.451696 1472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267d700c78264b30abf8a97244cee9a4. Latest consensus state: current_term: 1 leader_uuid: "267d700c78264b30abf8a97244cee9a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } } }
I20260924 16:10:59.451835 1472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:59.452265 1473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267d700c78264b30abf8a97244cee9a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267d700c78264b30abf8a97244cee9a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34779 } } }
I20260924 16:10:59.452364 1473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4 [sys.catalog]: This master's current role is: LEADER
I20260924 16:10:59.452817 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:10:59.448164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55503 (local address 127.0.0.1:34779)
0924 16:10:59.448274 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:10:59.448279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:10:59.448434 (+ 155us) server_negotiation.cc:407] Connection header received
0924 16:10:59.448571 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:10:59.448574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:10:59.448620 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:10:59.448704 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:10:59.448975 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:59.449672 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:59.450879 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:10:59.451191 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:10:59.451283 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:10:59.452363 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:10:59.452367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:10:59.452371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:10:59.452388 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:10:59.452414 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:10:59.452428 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:10:59.452511 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 16:10:59.452638 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 16:10:59.452673 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260924 16:10:59.455958 1487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:10:59.456076 1487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:10:59.456142 1481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:10:59.457064 1481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:10:59.462633 1481 catalog_manager.cc:1347] Generated new cluster ID: 1f0bad435bfc41efa3731692c58ef88a
I20260924 16:10:59.462706 1481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:10:59.476219 1481 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:10:59.477314 1481 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:10:59.488652 1481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 267d700c78264b30abf8a97244cee9a4: Generated new TSK 0
I20260924 16:10:59.489605 1481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:10:59.496253 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1392
W20260924 16:10:59.502619 1377 connection.cc:535] client connection to 127.0.0.1:34779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34779): Connection reset by peer (error 104)
2026-09-24T16:10:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:10:59 dist-test-slave-vj6n krb5kdc[1493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:10:59 dist-test-slave-vj6n krb5kdc[1493](info): set up 2 sockets
Sep 24 16:10:59 dist-test-slave-vj6n krb5kdc[1493](info): commencing operation
krb5kdc: starting...
W20260924 16:11:00.543362 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:00 dist-test-slave-vj6n krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:00.668773 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:00.782241 1506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:00.782538 1506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:00.782581 1506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:00.786531 1506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:00.786589 1506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:00.786610 1506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:00.786628 1506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:00.786643 1506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:00.791977 1506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:00.793059 1506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:00.794082 1506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:00.802745 1514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:00.802783 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:00.802745 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:00.802960 1506 server_base.cc:1034] running on GCE node
I20260924 16:11:00.803625 1506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:00.804836 1506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:00.806032 1506 hybrid_clock.cc:648] HybridClock initialized: now 1790266260806023 us; error 27 us; skew 500 ppm
Sep 24 16:11:00 dist-test-slave-vj6n krb5kdc[1493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:00.808893 1506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:00.809829 1506 webserver.cc:460] Webserver started at http://127.0.0.1:34761/ using document root <none> and password file <none>
I20260924 16:11:00.810493 1506 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:00.810590 1506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:00.810843 1506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:00.813000 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "a0d6da94ccd34ed784969c73ae2d7647"
format_stamp: "Formatted at 2026-09-24 16:11:00 on dist-test-slave-vj6n"
server_key: "de5cc68e9a1665ec65d884148afcd035"
server_key_iv: "48142a59db7c6fe9f7d32e48829fd810"
server_key_version: "encryptionkey@0"
I20260924 16:11:00.813612 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "a0d6da94ccd34ed784969c73ae2d7647"
format_stamp: "Formatted at 2026-09-24 16:11:00 on dist-test-slave-vj6n"
server_key: "de5cc68e9a1665ec65d884148afcd035"
server_key_iv: "48142a59db7c6fe9f7d32e48829fd810"
server_key_version: "encryptionkey@0"
I20260924 16:11:00.817689 1506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:11:00.820454 1521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:00.821743 1506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:11:00.821895 1506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "a0d6da94ccd34ed784969c73ae2d7647"
format_stamp: "Formatted at 2026-09-24 16:11:00 on dist-test-slave-vj6n"
server_key: "de5cc68e9a1665ec65d884148afcd035"
server_key_iv: "48142a59db7c6fe9f7d32e48829fd810"
server_key_version: "encryptionkey@0"
I20260924 16:11:00.822016 1506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:00.826903 1506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:00.829910 1506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:00.830127 1506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:00.837590 1506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41781
I20260924 16:11:00.837642 1573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41781 every 8 connection(s)
I20260924 16:11:00.838644 1506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:00.841704 1574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:00.844981 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1506
I20260924 16:11:00.845144 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:00.845432 13345 external_mini_cluster.cc:1428] Setting key f476eca4b03c4fc64ff2ae3ea0d6fa1f
I20260924 16:11:00.848486 1574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647: Bootstrap starting.
I20260924 16:11:00.851275 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:00.846932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59669 (local address 127.0.0.1:41781)
0924 16:11:00.847418 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:00.847440 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:00.847484 (+ 44us) server_negotiation.cc:407] Connection header received
0924 16:11:00.848308 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:00.848334 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:00.848733 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:00.849093 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:00.849844 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:00.849853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:00.849870 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:00.849885 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:00.849952 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:00.849979 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:00.850118 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:00.850188 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 16:11:00.850376 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":144,"threads_started":1}
I20260924 16:11:00.851348 1574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:00.852221 1574 log.cc:824] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:00.854408 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:00.854436 1574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647: No bootstrap required, opened a new log
I20260924 16:11:00.858178 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:00.856481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56735 (local address 127.0.0.1:41781)
0924 16:11:00.856619 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:00.856626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:00.856832 (+ 206us) server_negotiation.cc:407] Connection header received
0924 16:11:00.856884 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:00.856886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:00.856917 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:00.857002 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:00.857739 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:00.857741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:00.857743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:00.857754 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:00.857777 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:00.857788 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:00.857866 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:00.857970 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 16:11:00.858003 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 16:11:00.858565 1574 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } }
I20260924 16:11:00.858814 1574 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:00.858875 1574 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d6da94ccd34ed784969c73ae2d7647, State: Initialized, Role: FOLLOWER
I20260924 16:11:00.859347 1574 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } }
I20260924 16:11:00.859501 1574 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:00.859570 1574 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:00.859669 1574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:00.860585 1574 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } }
I20260924 16:11:00.860932 1574 leader_election.cc:302] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0d6da94ccd34ed784969c73ae2d7647; no voters:
I20260924 16:11:00.861248 1574 leader_election.cc:288] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:00.861404 1584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:00.861675 1584 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [term 1 LEADER]: Becoming Leader. State: Replica: a0d6da94ccd34ed784969c73ae2d7647, State: Running, Role: LEADER
I20260924 16:11:00.861992 1584 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } }
I20260924 16:11:00.862609 1574 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:00.863576 1584 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0d6da94ccd34ed784969c73ae2d7647. Latest consensus state: current_term: 1 leader_uuid: "a0d6da94ccd34ed784969c73ae2d7647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } } }
I20260924 16:11:00.864480 1584 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:00.864336 1586 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0d6da94ccd34ed784969c73ae2d7647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d6da94ccd34ed784969c73ae2d7647" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41781 } } }
I20260924 16:11:00.865108 1599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:00.865113 1586 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:00.868470 1599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:00.874475 1599 catalog_manager.cc:1347] Generated new cluster ID: f1eed9e79cbd40a982bc00a34f292480
I20260924 16:11:00.874554 1599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:00.888769 1599 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:00.889098 1599 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:00.898344 1599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0d6da94ccd34ed784969c73ae2d7647: Generated new TSK 0
I20260924 16:11:00.899078 1599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:00.905525 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1506
2026-09-24T16:11:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:00 dist-test-slave-vj6n krb5kdc[1606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:00 dist-test-slave-vj6n krb5kdc[1606](info): set up 2 sockets
Sep 24 16:11:00 dist-test-slave-vj6n krb5kdc[1606](info): commencing operation
krb5kdc: starting...
W20260924 16:11:02.005426 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:02.157963 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:02.274780 1619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:02.275091 1619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:02.275177 1619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:02.279251 1619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:02.279333 1619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:02.279372 1619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:02.279425 1619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:02.279469 1619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:02.284911 1619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:02.286039 1619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:02.287210 1619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:02.296017 1627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:02.296093 1625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:02.296031 1624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:02.296303 1619 server_base.cc:1034] running on GCE node
I20260924 16:11:02.296993 1619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:02.298164 1619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:02.299391 1619 hybrid_clock.cc:648] HybridClock initialized: now 1790266262299374 us; error 27 us; skew 500 ppm
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:02.308396 1619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:02.309357 1619 webserver.cc:460] Webserver started at http://127.0.0.1:34569/ using document root <none> and password file <none>
I20260924 16:11:02.309909 1619 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:02.309966 1619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:02.310144 1619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:02.312289 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "0a94e2d0521943e28f1642c5238abba3"
format_stamp: "Formatted at 2026-09-24 16:11:02 on dist-test-slave-vj6n"
server_key: "5b859a46ede772f431e02100bd267397"
server_key_iv: "5af624406024b18aae0c19dffa1c6cbd"
server_key_version: "encryptionkey@0"
I20260924 16:11:02.312834 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "0a94e2d0521943e28f1642c5238abba3"
format_stamp: "Formatted at 2026-09-24 16:11:02 on dist-test-slave-vj6n"
server_key: "5b859a46ede772f431e02100bd267397"
server_key_iv: "5af624406024b18aae0c19dffa1c6cbd"
server_key_version: "encryptionkey@0"
I20260924 16:11:02.316597 1619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:11:02.319038 1634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:02.320379 1619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:11:02.320477 1619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "0a94e2d0521943e28f1642c5238abba3"
format_stamp: "Formatted at 2026-09-24 16:11:02 on dist-test-slave-vj6n"
server_key: "5b859a46ede772f431e02100bd267397"
server_key_iv: "5af624406024b18aae0c19dffa1c6cbd"
server_key_version: "encryptionkey@0"
I20260924 16:11:02.320559 1619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:02.339666 1619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:02.342854 1619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:02.343024 1619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:02.350392 1619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45853
I20260924 16:11:02.350417 1686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45853 every 8 connection(s)
I20260924 16:11:02.351423 1619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:02.354445 1687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:02.354816 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1619
I20260924 16:11:02.354955 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:02.355252 13345 external_mini_cluster.cc:1428] Setting key 71afb06cc7cd58de1bca0b2a970c59bd
I20260924 16:11:02.362151 1687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3: Bootstrap starting.
I20260924 16:11:02.364754 1687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3: Neither blocks nor log segments found. Creating new log.
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 16:11:02.365734 1687 log.cc:824] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:02.367730 1687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3: No bootstrap required, opened a new log
I20260924 16:11:02.370524 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:02.356767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35351 (local address 127.0.0.1:45853)
0924 16:11:02.357218 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:02.357241 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:02.357287 (+ 46us) server_negotiation.cc:407] Connection header received
0924 16:11:02.358043 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:02.358069 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:02.358574 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:02.358969 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:02.359676 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:02.361210 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:02.362104 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:02.362806 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:02.362961 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:02.365770 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:02.365775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:02.365789 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:02.365844 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:02.367590 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:02.368083 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:02.368088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:02.368093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:02.368216 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:02.368524 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:02.368526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:02.368528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:02.368919 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:02.369056 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:02.369382 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 16:11:02.369697 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":110,"threads_started":1}
I20260924 16:11:02.372073 1687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260924 16:11:02.372352 1687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:02.372483 1687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a94e2d0521943e28f1642c5238abba3, State: Initialized, Role: FOLLOWER
I20260924 16:11:02.373013 1687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260924 16:11:02.373184 1687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:02.373250 1687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:02.373355 1687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:02.373492 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:02.374349 1687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260924 16:11:02.374692 1687 leader_election.cc:302] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a94e2d0521943e28f1642c5238abba3; no voters:
I20260924 16:11:02.375100 1687 leader_election.cc:288] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:02.375177 1696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:02.375411 1696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [term 1 LEADER]: Becoming Leader. State: Replica: 0a94e2d0521943e28f1642c5238abba3, State: Running, Role: LEADER
I20260924 16:11:02.375720 1696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260924 16:11:02.376516 1687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:02.377045 1696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a94e2d0521943e28f1642c5238abba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } } }
I20260924 16:11:02.377838 1696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:02.377717 1697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a94e2d0521943e28f1642c5238abba3. Latest consensus state: current_term: 1 leader_uuid: "0a94e2d0521943e28f1642c5238abba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a94e2d0521943e28f1642c5238abba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } } }
I20260924 16:11:02.378149 1697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:02.379212 1706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:02.382429 1706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:02.385006 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:02.375738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39873 (local address 127.0.0.1:45853)
0924 16:11:02.375853 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:02.375857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:02.376071 (+ 214us) server_negotiation.cc:407] Connection header received
0924 16:11:02.376257 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:02.376260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:02.376312 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:02.376395 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:02.377196 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:02.377983 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:02.379349 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:02.379666 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:02.379728 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:02.381425 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:02.381429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:02.381432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:02.381457 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:02.383178 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:02.383680 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:02.383683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:02.383685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:02.383776 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:02.384098 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:02.384102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:02.384103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:02.384320 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:02.384454 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:02.384726 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 16:11:02.384847 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 16:11:02.388504 1706 catalog_manager.cc:1347] Generated new cluster ID: a8b663bdcd8746e6a4db65b390d8e23e
I20260924 16:11:02.388571 1706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:02.403644 1706 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:02.404628 1706 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:02.410565 1706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a94e2d0521943e28f1642c5238abba3: Generated new TSK 0
I20260924 16:11:02.411283 1706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:02.421903 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1619
2026-09-24T16:11:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1718](info): set up 2 sockets
Sep 24 16:11:02 dist-test-slave-vj6n krb5kdc[1718](info): commencing operation
krb5kdc: starting...
W20260924 16:11:03.495299 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:03 dist-test-slave-vj6n krb5kdc[1718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:03.619681 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:03.733368 1732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:03.733636 1732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:03.733677 1732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:03.738250 1732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:03.738304 1732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:03.738322 1732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:03.738339 1732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:03.738354 1732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:03.743539 1732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:03.744539 1732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:03.745591 1732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:03.754415 1738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:03.754493 1732 server_base.cc:1034] running on GCE node
W20260924 16:11:03.754422 1739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:03.754566 1741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:03.755223 1732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:03.756366 1732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:03.757540 1732 hybrid_clock.cc:648] HybridClock initialized: now 1790266263757508 us; error 41 us; skew 500 ppm
Sep 24 16:11:03 dist-test-slave-vj6n krb5kdc[1718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:03.760540 1732 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:03.761505 1732 webserver.cc:460] Webserver started at http://127.0.0.1:34043/ using document root <none> and password file <none>
I20260924 16:11:03.762127 1732 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:03.762192 1732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:03.762460 1732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:03.764580 1732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "66e2136cc6bb47dfbcafc342512c753a"
format_stamp: "Formatted at 2026-09-24 16:11:03 on dist-test-slave-vj6n"
server_key: "c9623deb845a2948c3657723905ee7bc"
server_key_iv: "1a7f42630e3f65d989ec324573798d08"
server_key_version: "encryptionkey@0"
I20260924 16:11:03.765178 1732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "66e2136cc6bb47dfbcafc342512c753a"
format_stamp: "Formatted at 2026-09-24 16:11:03 on dist-test-slave-vj6n"
server_key: "c9623deb845a2948c3657723905ee7bc"
server_key_iv: "1a7f42630e3f65d989ec324573798d08"
server_key_version: "encryptionkey@0"
I20260924 16:11:03.769091 1732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260924 16:11:03.771656 1748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:03.772943 1732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:11:03.773113 1732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "66e2136cc6bb47dfbcafc342512c753a"
format_stamp: "Formatted at 2026-09-24 16:11:03 on dist-test-slave-vj6n"
server_key: "c9623deb845a2948c3657723905ee7bc"
server_key_iv: "1a7f42630e3f65d989ec324573798d08"
server_key_version: "encryptionkey@0"
I20260924 16:11:03.773231 1732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:03.788177 1732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:03.791172 1732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:03.791393 1732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:03.799113 1732 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35637
I20260924 16:11:03.799142 1800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35637 every 8 connection(s)
I20260924 16:11:03.800088 1732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:03.802958 1801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:03.806376 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1732
I20260924 16:11:03.806531 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:03.806803 13345 external_mini_cluster.cc:1428] Setting key e34817c1ae700362e94f5d09ba74cd96
I20260924 16:11:03.809206 1801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a: Bootstrap starting.
I20260924 16:11:03.811512 1801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:03.812268 1801 log.cc:824] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:03.814314 1801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a: No bootstrap required, opened a new log
I20260924 16:11:03.816183 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:03.808368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47465 (local address 127.0.0.1:35637)
0924 16:11:03.808823 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:03.808842 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:03.808879 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:11:03.809572 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:03.809591 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:03.809952 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:03.810304 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:03.810732 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:03.811776 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:03.812744 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:03.813213 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:03.813385 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:03.814275 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:03.814281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:03.814303 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:03.814736 (+ 433us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:03.814790 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:03.814814 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:03.814921 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:03.815089 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 16:11:03.815297 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260924 16:11:03.817401 1801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260924 16:11:03.817652 1801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:03.817734 1801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e2136cc6bb47dfbcafc342512c753a, State: Initialized, Role: FOLLOWER
I20260924 16:11:03.818207 1801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260924 16:11:03.818423 1801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:03.818506 1801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:03.818614 1801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:03.819016 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:03.819566 1801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260924 16:11:03.819939 1801 leader_election.cc:302] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e2136cc6bb47dfbcafc342512c753a; no voters:
I20260924 16:11:03.820274 1801 leader_election.cc:288] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:03.820340 1807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:03.820537 1807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [term 1 LEADER]: Becoming Leader. State: Replica: 66e2136cc6bb47dfbcafc342512c753a, State: Running, Role: LEADER
I20260924 16:11:03.820848 1807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260924 16:11:03.821695 1801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:03.822083 1807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e2136cc6bb47dfbcafc342512c753a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } } }
I20260924 16:11:03.822196 1807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:03.822580 1811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e2136cc6bb47dfbcafc342512c753a. Latest consensus state: current_term: 1 leader_uuid: "66e2136cc6bb47dfbcafc342512c753a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e2136cc6bb47dfbcafc342512c753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } } }
I20260924 16:11:03.822762 1811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:03.824009 1814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:03.827116 1814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:03.828320 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:03.822787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34701 (local address 127.0.0.1:35637)
0924 16:11:03.822897 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:03.822904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:03.823396 (+ 492us) server_negotiation.cc:407] Connection header received
0924 16:11:03.824271 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:03.824274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:03.824320 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:03.824397 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:03.824701 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:03.825420 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:03.826570 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:03.826877 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:03.826980 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:03.827875 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:03.827878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:03.827881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:03.827893 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:03.827920 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:03.827933 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:03.828008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:03.828149 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 16:11:03.828200 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 16:11:03.832607 1814 catalog_manager.cc:1347] Generated new cluster ID: 588f1197b3354b3ab954c9863c28b437
I20260924 16:11:03.832680 1814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:03.864847 1814 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:03.865873 1814 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:03.874769 1814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e2136cc6bb47dfbcafc342512c753a: Generated new TSK 0
I20260924 16:11:03.875511 1814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:03.883527 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1732
2026-09-24T16:11:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:03 dist-test-slave-vj6n krb5kdc[1832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:03 dist-test-slave-vj6n krb5kdc[1832](info): set up 2 sockets
Sep 24 16:11:03 dist-test-slave-vj6n krb5kdc[1832](info): commencing operation
krb5kdc: starting...
W20260924 16:11:04.931505 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:05 dist-test-slave-vj6n krb5kdc[1832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:05.053921 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:05.169565 1845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:05.169875 1845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:05.169956 1845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:05.174052 1845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:05.174156 1845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:05.174191 1845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:05.174252 1845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:05.174289 1845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:05.179822 1845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:05.180970 1845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:05.182057 1845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:05.190613 1850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:05.190670 1853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:05.190603 1851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:05.190867 1845 server_base.cc:1034] running on GCE node
I20260924 16:11:05.191383 1845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:05.192490 1845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:05.193657 1845 hybrid_clock.cc:648] HybridClock initialized: now 1790266265193638 us; error 29 us; skew 500 ppm
Sep 24 16:11:05 dist-test-slave-vj6n krb5kdc[1832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:05.196318 1845 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:05.197192 1845 webserver.cc:460] Webserver started at http://127.0.0.1:39985/ using document root <none> and password file <none>
I20260924 16:11:05.197723 1845 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:05.197780 1845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:05.197939 1845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:05.199998 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e"
format_stamp: "Formatted at 2026-09-24 16:11:05 on dist-test-slave-vj6n"
server_key: "83dd733353fd3d1e978c3f49c575001e"
server_key_iv: "0af4e79807f4604cb509479ba095fb19"
server_key_version: "encryptionkey@0"
I20260924 16:11:05.200474 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e"
format_stamp: "Formatted at 2026-09-24 16:11:05 on dist-test-slave-vj6n"
server_key: "83dd733353fd3d1e978c3f49c575001e"
server_key_iv: "0af4e79807f4604cb509479ba095fb19"
server_key_version: "encryptionkey@0"
I20260924 16:11:05.204169 1845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:11:05.206597 1860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:05.207844 1845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:11:05.207949 1845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e"
format_stamp: "Formatted at 2026-09-24 16:11:05 on dist-test-slave-vj6n"
server_key: "83dd733353fd3d1e978c3f49c575001e"
server_key_iv: "0af4e79807f4604cb509479ba095fb19"
server_key_version: "encryptionkey@0"
I20260924 16:11:05.208035 1845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:05.212615 1845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:05.215468 1845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:05.215610 1845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:05.222828 1845 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35977
I20260924 16:11:05.222856 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35977 every 8 connection(s)
I20260924 16:11:05.223764 1845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:05.226660 1914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:05.230111 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1845
I20260924 16:11:05.230333 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:05.230604 13345 external_mini_cluster.cc:1428] Setting key a9f7591979d71734bda61563ef5f2a34
I20260924 16:11:05.232693 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e: Bootstrap starting.
I20260924 16:11:05.235224 1914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:05.235981 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:05.231964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43563 (local address 127.0.0.1:35977)
0924 16:11:05.232456 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:05.232470 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:05.232506 (+ 36us) server_negotiation.cc:407] Connection header received
0924 16:11:05.233212 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:05.233231 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:05.233526 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:05.233880 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:05.234702 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:05.234712 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:05.234725 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:05.234735 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:05.234798 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:05.234816 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:05.234933 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:05.235060 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 16:11:05.235230 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":144,"threads_started":1}
I20260924 16:11:05.236024 1914 log.cc:824] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:05.238066 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e: No bootstrap required, opened a new log
I20260924 16:11:05.239066 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:05.242478 1914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 16:11:05.242751 1914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:05.242856 1914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1adbc23ae0f642f9b1501ae6b3f13a3e, State: Initialized, Role: FOLLOWER
I20260924 16:11:05.243018 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:05.241195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45087 (local address 127.0.0.1:35977)
0924 16:11:05.241339 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:05.241344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:05.241525 (+ 181us) server_negotiation.cc:407] Connection header received
0924 16:11:05.241680 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:05.241683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:05.241723 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:05.241807 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:05.242544 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:05.242548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:05.242551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:05.242569 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:05.242595 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:05.242605 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:05.242698 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:05.242841 (+ 143us) server_negotiation.cc:299] Negotiation successful
0924 16:11:05.242879 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 16:11:05.243456 1914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 16:11:05.243642 1914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:05.243712 1914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:05.243814 1914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:05.244791 1914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 16:11:05.245106 1914 leader_election.cc:302] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1adbc23ae0f642f9b1501ae6b3f13a3e; no voters:
I20260924 16:11:05.245421 1914 leader_election.cc:288] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:05.245548 1924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:05.245802 1924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [term 1 LEADER]: Becoming Leader. State: Replica: 1adbc23ae0f642f9b1501ae6b3f13a3e, State: Running, Role: LEADER
I20260924 16:11:05.246170 1924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } }
I20260924 16:11:05.246892 1914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:05.248098 1926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1adbc23ae0f642f9b1501ae6b3f13a3e. Latest consensus state: current_term: 1 leader_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } } }
I20260924 16:11:05.248243 1926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:05.248533 1925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adbc23ae0f642f9b1501ae6b3f13a3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35977 } } }
I20260924 16:11:05.248631 1933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:05.249774 1925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:05.252249 1933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:05.257875 1933 catalog_manager.cc:1347] Generated new cluster ID: 7a8403ed3de441f6b947a59b88bebe50
I20260924 16:11:05.257942 1933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:05.277020 1933 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:05.277422 1933 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:05.283116 1933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1adbc23ae0f642f9b1501ae6b3f13a3e: Generated new TSK 0
I20260924 16:11:05.283874 1933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:05.290189 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1845
2026-09-24T16:11:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:05 dist-test-slave-vj6n krb5kdc[1945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:05 dist-test-slave-vj6n krb5kdc[1945](info): set up 2 sockets
Sep 24 16:11:05 dist-test-slave-vj6n krb5kdc[1945](info): commencing operation
krb5kdc: starting...
W20260924 16:11:06.339408 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:06.461933 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42991
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:06.575143 1958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:06.575471 1958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:06.575544 1958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:06.579651 1958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:06.579746 1958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:06.579785 1958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:06.579843 1958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:06.579882 1958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:06.585397 1958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:06.586615 1958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:06.587709 1958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:06.596961 1963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:06.596925 1966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:06.597005 1958 server_base.cc:1034] running on GCE node
W20260924 16:11:06.596951 1964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:06.597786 1958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:06.599005 1958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:06.600204 1958 hybrid_clock.cc:648] HybridClock initialized: now 1790266266600184 us; error 30 us; skew 500 ppm
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[1945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:06.603138 1958 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:06.604092 1958 webserver.cc:460] Webserver started at http://127.0.0.1:42865/ using document root <none> and password file <none>
I20260924 16:11:06.604661 1958 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:06.604722 1958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:06.604897 1958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:06.607100 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "414210b25cc544ac9273af4f14b5c825"
format_stamp: "Formatted at 2026-09-24 16:11:06 on dist-test-slave-vj6n"
server_key: "00bb9bbdd89b8375dc1a070a51c968e9"
server_key_iv: "fbcf84d17e30e560ebc30674ae57701e"
server_key_version: "encryptionkey@0"
I20260924 16:11:06.607636 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "414210b25cc544ac9273af4f14b5c825"
format_stamp: "Formatted at 2026-09-24 16:11:06 on dist-test-slave-vj6n"
server_key: "00bb9bbdd89b8375dc1a070a51c968e9"
server_key_iv: "fbcf84d17e30e560ebc30674ae57701e"
server_key_version: "encryptionkey@0"
I20260924 16:11:06.611548 1958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260924 16:11:06.614053 1973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:06.615468 1958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:11:06.615574 1958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "414210b25cc544ac9273af4f14b5c825"
format_stamp: "Formatted at 2026-09-24 16:11:06 on dist-test-slave-vj6n"
server_key: "00bb9bbdd89b8375dc1a070a51c968e9"
server_key_iv: "fbcf84d17e30e560ebc30674ae57701e"
server_key_version: "encryptionkey@0"
I20260924 16:11:06.615671 1958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:06.641692 1958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:06.645149 1958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:06.645309 1958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:06.652884 1958 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42991
I20260924 16:11:06.652976 2025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42991 every 8 connection(s)
I20260924 16:11:06.654069 1958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:06.657034 2026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:06.658587 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 1958
I20260924 16:11:06.658733 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:06.659003 13345 external_mini_cluster.cc:1428] Setting key 2a91b197f2b1a95ff6302d207be342c3
I20260924 16:11:06.663542 2026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: Bootstrap starting.
I20260924 16:11:06.665819 2026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:06.666718 2026 log.cc:824] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: Log is configured to *not* fsync() on all Append() calls
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[1945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 16:11:06.668968 2026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: No bootstrap required, opened a new log
I20260924 16:11:06.672677 2026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } }
I20260924 16:11:06.672987 2026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:06.673081 2026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 414210b25cc544ac9273af4f14b5c825, State: Initialized, Role: FOLLOWER
I20260924 16:11:06.673547 2026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } }
I20260924 16:11:06.673671 2026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:06.673753 2026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:06.673866 2026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:06.674000 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:06.660467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:46819 (local address 127.0.0.1:42991)
0924 16:11:06.660940 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:06.660955 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:06.660993 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:11:06.661628 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:06.661656 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:06.662015 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:06.662389 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:06.663305 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:06.664365 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:06.665275 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:06.665758 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:06.665940 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:06.668931 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:06.668941 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:06.668955 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:06.668982 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:06.670736 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:06.671269 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:06.671274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:06.671279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:06.671389 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:06.671743 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:06.671751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:06.671753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:06.672153 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:06.672390 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:06.672693 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 16:11:06.672917 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":142,"threads_started":1}
I20260924 16:11:06.674867 2026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } }
I20260924 16:11:06.675242 2026 leader_election.cc:302] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 414210b25cc544ac9273af4f14b5c825; no voters:
I20260924 16:11:06.675585 2026 leader_election.cc:288] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:06.675719 2031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:06.675933 2031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [term 1 LEADER]: Becoming Leader. State: Replica: 414210b25cc544ac9273af4f14b5c825, State: Running, Role: LEADER
I20260924 16:11:06.676347 2031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } }
I20260924 16:11:06.676862 2026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:06.677508 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:06.678085 2033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 414210b25cc544ac9273af4f14b5c825. Latest consensus state: current_term: 1 leader_uuid: "414210b25cc544ac9273af4f14b5c825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } } }
I20260924 16:11:06.678234 2033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:06.678594 2032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "414210b25cc544ac9273af4f14b5c825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414210b25cc544ac9273af4f14b5c825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42991 } } }
I20260924 16:11:06.678674 2032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825 [sys.catalog]: This master's current role is: LEADER
W20260924 16:11:06.681922 2047 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:06.682003 2047 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:06.682387 2040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:06.683167 2040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:06.689539 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:06.679857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:48605 (local address 127.0.0.1:42991)
0924 16:11:06.679978 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:06.679983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:06.680399 (+ 416us) server_negotiation.cc:407] Connection header received
0924 16:11:06.680527 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:06.680531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:06.680589 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:06.680678 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:06.681504 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:06.682288 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:06.683417 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:06.683756 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:06.683863 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:06.685588 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:06.685591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:06.685594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:06.685620 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:06.687470 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:06.688045 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:06.688049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:06.688051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:06.688146 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:06.688501 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:06.688504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:06.688506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:06.688770 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:06.688929 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:06.689260 (+ 331us) server_negotiation.cc:299] Negotiation successful
0924 16:11:06.689372 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 16:11:06.689898 2040 catalog_manager.cc:1347] Generated new cluster ID: b8f03e6d25d64fecb6bc1b471aaef199
I20260924 16:11:06.689960 2040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:06.696254 2040 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:06.697419 2040 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:06.709961 2040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 414210b25cc544ac9273af4f14b5c825: Generated new TSK 0
I20260924 16:11:06.710749 2040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:06.715862 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 1958
W20260924 16:11:06.722198 1943 connection.cc:535] client connection to 127.0.0.1:42991 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42991): Connection reset by peer (error 104)
2026-09-24T16:11:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[2058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[2058](info): set up 2 sockets
Sep 24 16:11:06 dist-test-slave-vj6n krb5kdc[2058](info): commencing operation
krb5kdc: starting...
W20260924 16:11:07.795370 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:07 dist-test-slave-vj6n krb5kdc[2058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:07.921028 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:08.036847 2071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:08.037134 2071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:08.037178 2071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:08.041286 2071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:08.041368 2071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:08.041389 2071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:08.041407 2071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:08.041424 2071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:08.046918 2071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:08.048054 2071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:08.049093 2071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:08.058027 2078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:08.058265 2071 server_base.cc:1034] running on GCE node
W20260924 16:11:08.058040 2080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:08.058033 2077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:08.058931 2071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:08.060113 2071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:08.061309 2071 hybrid_clock.cc:648] HybridClock initialized: now 1790266268061289 us; error 29 us; skew 500 ppm
Sep 24 16:11:08 dist-test-slave-vj6n krb5kdc[2058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:08.069783 2071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:08.070978 2071 webserver.cc:460] Webserver started at http://127.0.0.1:37213/ using document root <none> and password file <none>
I20260924 16:11:08.071610 2071 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:08.071710 2071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:08.071959 2071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:08.074131 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "07628ea2d03a497a87d8e4517cd10ceb"
format_stamp: "Formatted at 2026-09-24 16:11:08 on dist-test-slave-vj6n"
server_key: "d8877afc397442981e6cf8de07568174"
server_key_iv: "d262beb1f9fc0b7870873ae8a6076966"
server_key_version: "encryptionkey@0"
I20260924 16:11:08.074775 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "07628ea2d03a497a87d8e4517cd10ceb"
format_stamp: "Formatted at 2026-09-24 16:11:08 on dist-test-slave-vj6n"
server_key: "d8877afc397442981e6cf8de07568174"
server_key_iv: "d262beb1f9fc0b7870873ae8a6076966"
server_key_version: "encryptionkey@0"
I20260924 16:11:08.078634 2071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:11:08.081138 2087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:08.082546 2071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:11:08.082720 2071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "07628ea2d03a497a87d8e4517cd10ceb"
format_stamp: "Formatted at 2026-09-24 16:11:08 on dist-test-slave-vj6n"
server_key: "d8877afc397442981e6cf8de07568174"
server_key_iv: "d262beb1f9fc0b7870873ae8a6076966"
server_key_version: "encryptionkey@0"
I20260924 16:11:08.082847 2071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:08.120261 2071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:08.123417 2071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:08.123643 2071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:08.131999 2071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33585
I20260924 16:11:08.132031 2139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33585 every 8 connection(s)
I20260924 16:11:08.133056 2071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:08.136173 2140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:08.138573 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2071
I20260924 16:11:08.138711 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:08.139027 13345 external_mini_cluster.cc:1428] Setting key f2ad50d6135e68b23446d2f42d7cab5e
I20260924 16:11:08.143003 2140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb: Bootstrap starting.
I20260924 16:11:08.145618 2140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:08.146543 2140 log.cc:824] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:08.148705 2140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb: No bootstrap required, opened a new log
I20260924 16:11:08.148833 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:08.140571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:43977 (local address 127.0.0.1:33585)
0924 16:11:08.141047 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:08.141063 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:08.141101 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:11:08.141996 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:08.142016 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:08.142489 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:08.142885 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:08.143349 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:08.144408 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:08.145331 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:08.145887 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:08.146080 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:08.147063 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:08.147068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:08.147082 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:08.147475 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:08.147541 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:08.147564 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:08.147709 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:08.147901 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 16:11:08.148061 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
I20260924 16:11:08.152200 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:08.153158 2140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260924 16:11:08.153496 2140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:08.153571 2140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07628ea2d03a497a87d8e4517cd10ceb, State: Initialized, Role: FOLLOWER
I20260924 16:11:08.154088 2140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260924 16:11:08.154304 2140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:08.154389 2140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:08.154515 2140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:08.155543 2140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260924 16:11:08.155923 2140 leader_election.cc:302] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07628ea2d03a497a87d8e4517cd10ceb; no voters:
I20260924 16:11:08.156309 2140 leader_election.cc:288] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:08.156394 2151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:08.156634 2151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [term 1 LEADER]: Becoming Leader. State: Replica: 07628ea2d03a497a87d8e4517cd10ceb, State: Running, Role: LEADER
I20260924 16:11:08.157001 2151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260924 16:11:08.157805 2140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:08.158842 2153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07628ea2d03a497a87d8e4517cd10ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } } }
I20260924 16:11:08.158989 2153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:08.159010 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:08.154425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:59151 (local address 127.0.0.1:33585)
0924 16:11:08.154574 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:08.154578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:08.154725 (+ 147us) server_negotiation.cc:407] Connection header received
0924 16:11:08.154885 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:08.154888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:08.154945 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:08.155050 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:08.155330 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:08.155927 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:08.156967 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:08.157270 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:08.157340 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:08.158428 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:08.158432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:08.158435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:08.158449 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:08.158473 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:08.158488 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:08.158567 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:08.158815 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 16:11:08.158857 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:11:08.160386 2160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:08.162364 2152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07628ea2d03a497a87d8e4517cd10ceb. Latest consensus state: current_term: 1 leader_uuid: "07628ea2d03a497a87d8e4517cd10ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07628ea2d03a497a87d8e4517cd10ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } } }
I20260924 16:11:08.162655 2152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:08.163724 2160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:08.169236 2160 catalog_manager.cc:1347] Generated new cluster ID: cd70cced4c7e4e59a336b6b855c3ff7c
I20260924 16:11:08.169304 2160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:08.184055 2160 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:08.185107 2160 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:08.196995 2160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07628ea2d03a497a87d8e4517cd10ceb: Generated new TSK 0
I20260924 16:11:08.197773 2160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:08.205829 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2071
W20260924 16:11:08.212059 2056 connection.cc:535] client connection to 127.0.0.1:33585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33585): Connection reset by peer (error 104)
2026-09-24T16:11:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:08 dist-test-slave-vj6n krb5kdc[2172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:08 dist-test-slave-vj6n krb5kdc[2172](info): set up 2 sockets
Sep 24 16:11:08 dist-test-slave-vj6n krb5kdc[2172](info): commencing operation
krb5kdc: starting...
W20260924 16:11:09.246560 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:09 dist-test-slave-vj6n krb5kdc[2172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:09.379508 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:09.494112 2185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:09.494441 2185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:09.494578 2185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:09.499163 2185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:09.499244 2185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:09.499265 2185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:09.499282 2185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:09.499336 2185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:09.504954 2185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:09.506167 2185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:09.507342 2185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:09.516053 2190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:09.516194 2185 server_base.cc:1034] running on GCE node
W20260924 16:11:09.516305 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:09.516341 2191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:09.517112 2185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:09.518324 2185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:09.519538 2185 hybrid_clock.cc:648] HybridClock initialized: now 1790266269519523 us; error 37 us; skew 500 ppm
Sep 24 16:11:09 dist-test-slave-vj6n krb5kdc[2172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:09.522526 2185 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:09.523468 2185 webserver.cc:460] Webserver started at http://127.0.0.1:33325/ using document root <none> and password file <none>
I20260924 16:11:09.524106 2185 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:09.524204 2185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:09.524456 2185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:09.526737 2185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "94aff68409dc4a53a5bd2ef36301822f"
format_stamp: "Formatted at 2026-09-24 16:11:09 on dist-test-slave-vj6n"
server_key: "0476880560ba4384e2178511661bf3b8"
server_key_iv: "3798a18a2ad359d2ad3064fb7ed9ce5b"
server_key_version: "encryptionkey@0"
I20260924 16:11:09.527328 2185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "94aff68409dc4a53a5bd2ef36301822f"
format_stamp: "Formatted at 2026-09-24 16:11:09 on dist-test-slave-vj6n"
server_key: "0476880560ba4384e2178511661bf3b8"
server_key_iv: "3798a18a2ad359d2ad3064fb7ed9ce5b"
server_key_version: "encryptionkey@0"
I20260924 16:11:09.531354 2185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:11:09.533984 2200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:09.535406 2185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 16:11:09.535548 2185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "94aff68409dc4a53a5bd2ef36301822f"
format_stamp: "Formatted at 2026-09-24 16:11:09 on dist-test-slave-vj6n"
server_key: "0476880560ba4384e2178511661bf3b8"
server_key_iv: "3798a18a2ad359d2ad3064fb7ed9ce5b"
server_key_version: "encryptionkey@0"
I20260924 16:11:09.535683 2185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:09.541249 2185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:09.544351 2185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:09.544554 2185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:09.551703 2185 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46105
I20260924 16:11:09.551744 2252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46105 every 8 connection(s)
I20260924 16:11:09.552649 2185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:09.555435 2253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:09.556277 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2185
I20260924 16:11:09.556416 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:09.556703 13345 external_mini_cluster.cc:1428] Setting key 2e5ca22f4a9069aec83daf3b4c31d992
I20260924 16:11:09.561846 2253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f: Bootstrap starting.
I20260924 16:11:09.562023 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:09.558131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:47487 (local address 127.0.0.1:46105)
0924 16:11:09.558615 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:09.558630 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:09.558673 (+ 43us) server_negotiation.cc:407] Connection header received
0924 16:11:09.559284 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:09.559301 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:09.559579 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:09.559926 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:09.560790 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:09.560794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:09.560809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:09.560817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:09.560867 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:09.560885 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:09.561039 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:09.561123 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 16:11:09.561287 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":140,"threads_started":1}
I20260924 16:11:09.564535 2253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:09.564890 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:09.565371 2253 log.cc:824] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:09.567386 2253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f: No bootstrap required, opened a new log
I20260924 16:11:09.568702 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:09.566992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:34259 (local address 127.0.0.1:46105)
0924 16:11:09.567122 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:09.567129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:09.567318 (+ 189us) server_negotiation.cc:407] Connection header received
0924 16:11:09.567437 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:09.567442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:09.567510 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:09.567605 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:09.568295 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:09.568299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:09.568301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:09.568313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:09.568338 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:09.568357 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:09.568446 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:09.568519 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 16:11:09.568560 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 16:11:09.571749 2253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260924 16:11:09.572082 2253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:09.572165 2253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94aff68409dc4a53a5bd2ef36301822f, State: Initialized, Role: FOLLOWER
I20260924 16:11:09.572650 2253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260924 16:11:09.572779 2253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:09.572865 2253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:09.572988 2253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:09.573894 2253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260924 16:11:09.574280 2253 leader_election.cc:302] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94aff68409dc4a53a5bd2ef36301822f; no voters:
I20260924 16:11:09.574625 2253 leader_election.cc:288] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:09.574759 2263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:09.574990 2263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [term 1 LEADER]: Becoming Leader. State: Replica: 94aff68409dc4a53a5bd2ef36301822f, State: Running, Role: LEADER
I20260924 16:11:09.575347 2263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260924 16:11:09.576025 2253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:09.577266 2264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94aff68409dc4a53a5bd2ef36301822f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } } }
I20260924 16:11:09.577418 2264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:09.577819 2272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:09.577759 2265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94aff68409dc4a53a5bd2ef36301822f. Latest consensus state: current_term: 1 leader_uuid: "94aff68409dc4a53a5bd2ef36301822f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94aff68409dc4a53a5bd2ef36301822f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } } }
I20260924 16:11:09.578472 2265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:09.580929 2272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:09.586983 2272 catalog_manager.cc:1347] Generated new cluster ID: 569aa9497fcf49b29e7cf33d6e6cc89b
I20260924 16:11:09.587050 2272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:09.595317 2272 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:09.595659 2272 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:09.607064 2272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94aff68409dc4a53a5bd2ef36301822f: Generated new TSK 0
I20260924 16:11:09.607928 2272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:09.622696 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2185
2026-09-24T16:11:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:09 dist-test-slave-vj6n krb5kdc[2284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:09 dist-test-slave-vj6n krb5kdc[2284](info): set up 2 sockets
Sep 24 16:11:09 dist-test-slave-vj6n krb5kdc[2284](info): commencing operation
krb5kdc: starting...
W20260924 16:11:10.676471 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:10 dist-test-slave-vj6n krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:10.801185 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:10.915017 2298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:10.915323 2298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:10.915426 2298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:10.919517 2298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:10.919600 2298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:10.919636 2298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:10.919696 2298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:10.919734 2298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:10.925117 2298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:10.926308 2298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:10.927366 2298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:10.935904 2307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:10.936004 2298 server_base.cc:1034] running on GCE node
W20260924 16:11:10.935943 2304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:10.935904 2305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:10.936830 2298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:10.938042 2298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:10.939242 2298 hybrid_clock.cc:648] HybridClock initialized: now 1790266270939225 us; error 40 us; skew 500 ppm
Sep 24 16:11:10 dist-test-slave-vj6n krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:10.942294 2298 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:10.943250 2298 webserver.cc:460] Webserver started at http://127.0.0.1:40087/ using document root <none> and password file <none>
I20260924 16:11:10.943835 2298 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:10.943928 2298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:10.944164 2298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:10.946213 2298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "4cb77325bddb48ad84c56d8f244ae358"
format_stamp: "Formatted at 2026-09-24 16:11:10 on dist-test-slave-vj6n"
server_key: "f8bf420ed96a46bcb1d1a3aea409359c"
server_key_iv: "5af30723ee7217c56a7d39f73fb1f19f"
server_key_version: "encryptionkey@0"
I20260924 16:11:10.946887 2298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "4cb77325bddb48ad84c56d8f244ae358"
format_stamp: "Formatted at 2026-09-24 16:11:10 on dist-test-slave-vj6n"
server_key: "f8bf420ed96a46bcb1d1a3aea409359c"
server_key_iv: "5af30723ee7217c56a7d39f73fb1f19f"
server_key_version: "encryptionkey@0"
I20260924 16:11:10.950604 2298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:10.953066 2314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:10.954537 2298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:10.954675 2298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "4cb77325bddb48ad84c56d8f244ae358"
format_stamp: "Formatted at 2026-09-24 16:11:10 on dist-test-slave-vj6n"
server_key: "f8bf420ed96a46bcb1d1a3aea409359c"
server_key_iv: "5af30723ee7217c56a7d39f73fb1f19f"
server_key_version: "encryptionkey@0"
I20260924 16:11:10.954770 2298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:10.969025 2298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:10.972108 2298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:10.972275 2298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:10.979689 2298 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34605
I20260924 16:11:10.979779 2366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34605 every 8 connection(s)
I20260924 16:11:10.980731 2298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:10.983700 2367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:10.987432 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2298
I20260924 16:11:10.987599 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:10.987880 13345 external_mini_cluster.cc:1428] Setting key d2956824f3406c969bfb89848e231fb6
I20260924 16:11:10.990082 2367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: Bootstrap starting.
I20260924 16:11:10.992794 2367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:10.993654 2367 log.cc:824] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:10.995960 2367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: No bootstrap required, opened a new log
Sep 24 16:11:10 dist-test-slave-vj6n krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 16:11:10.999725 2367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260924 16:11:10.999946 2367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:11.000023 2367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb77325bddb48ad84c56d8f244ae358, State: Initialized, Role: FOLLOWER
I20260924 16:11:11.000543 2367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260924 16:11:11.000667 2367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:11.000743 2367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:11.000854 2367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:11.002072 2367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260924 16:11:11.002458 2367 leader_election.cc:302] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb77325bddb48ad84c56d8f244ae358; no voters:
I20260924 16:11:11.002818 2367 leader_election.cc:288] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:11.002940 2372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:11.003180 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:10.989285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:44445 (local address 127.0.0.1:34605)
0924 16:11:10.989716 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:10.989732 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:10.989770 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:11:10.990544 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:10.990566 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:10.990957 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:10.991248 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:10.992190 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:10.993267 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:10.994458 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:10.994945 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:10.995063 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:10.997845 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:10.997850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:10.997864 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:10.997891 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:10.999668 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:11.000299 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:11.000304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:11.000309 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:11.000421 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:11.000837 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:11.000841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:11.000843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:11.001378 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:11.001646 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:11.001878 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 16:11:11.002124 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260924 16:11:11.003189 2372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [term 1 LEADER]: Becoming Leader. State: Replica: 4cb77325bddb48ad84c56d8f244ae358, State: Running, Role: LEADER
I20260924 16:11:11.003729 2372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260924 16:11:11.004186 2367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:11.005373 2373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb77325bddb48ad84c56d8f244ae358. Latest consensus state: current_term: 1 leader_uuid: "4cb77325bddb48ad84c56d8f244ae358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } } }
I20260924 16:11:11.005487 2373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:11.006193 2374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb77325bddb48ad84c56d8f244ae358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb77325bddb48ad84c56d8f244ae358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } } }
I20260924 16:11:11.006985 2374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:11.007239 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260924 16:11:11.009711 2387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:11.009771 2387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:11.009855 2381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:11.010761 2381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:11.016024 2381 catalog_manager.cc:1347] Generated new cluster ID: 8cc1848f09444f448cdc5a0f62bc6e29
I20260924 16:11:11.016084 2381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:11.023729 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:11.010686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:49295 (local address 127.0.0.1:34605)
0924 16:11:11.010806 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:11.010811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:11.014593 (+ 3782us) server_negotiation.cc:407] Connection header received
0924 16:11:11.014796 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:11.014799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:11.014852 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:11.014960 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:11.015823 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:11.016641 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:11.017849 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:11.018193 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:11.018322 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:11.020185 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:11.020188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:11.020191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:11.020215 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:11.021891 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:11.022436 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:11.022440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:11.022442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:11.022523 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:11.022865 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:11.022868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:11.022870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:11.023058 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:11.023183 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:11.023477 (+ 294us) server_negotiation.cc:299] Negotiation successful
0924 16:11:11.023569 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:11:11.029929 2381 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:11.030963 2381 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:11.036983 2381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cb77325bddb48ad84c56d8f244ae358: Generated new TSK 0
I20260924 16:11:11.037703 2381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:11.046835 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2298
W20260924 16:11:11.053148 2282 connection.cc:535] client connection to 127.0.0.1:34605 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34605): Connection reset by peer (error 104)
2026-09-24T16:11:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:11 dist-test-slave-vj6n krb5kdc[2398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:11 dist-test-slave-vj6n krb5kdc[2398](info): set up 2 sockets
Sep 24 16:11:11 dist-test-slave-vj6n krb5kdc[2398](info): commencing operation
krb5kdc: starting...
W20260924 16:11:12.143508 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:12 dist-test-slave-vj6n krb5kdc[2398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:12.273761 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:12.389752 2411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:12.390046 2411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:12.390090 2411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:12.394209 2411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:12.394295 2411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:12.394315 2411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:12.394332 2411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:12.394392 2411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:12.401057 2411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:12.402829 2411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:12.404451 2411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:12.415035 2417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:12.415009 2416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:12.415158 2411 server_base.cc:1034] running on GCE node
W20260924 16:11:12.415005 2419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:12.415894 2411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:12.417099 2411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:12.418329 2411 hybrid_clock.cc:648] HybridClock initialized: now 1790266272418292 us; error 41 us; skew 500 ppm
Sep 24 16:11:12 dist-test-slave-vj6n krb5kdc[2398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:12.421687 2411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:12.422780 2411 webserver.cc:460] Webserver started at http://127.0.0.1:45265/ using document root <none> and password file <none>
I20260924 16:11:12.423388 2411 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:12.423483 2411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:12.423728 2411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:12.425823 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6"
format_stamp: "Formatted at 2026-09-24 16:11:12 on dist-test-slave-vj6n"
server_key: "11269a71e03ec60813146acd6d762fed"
server_key_iv: "916b4fea2489f28bead86c4b6f756e2f"
server_key_version: "encryptionkey@0"
I20260924 16:11:12.426469 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6"
format_stamp: "Formatted at 2026-09-24 16:11:12 on dist-test-slave-vj6n"
server_key: "11269a71e03ec60813146acd6d762fed"
server_key_iv: "916b4fea2489f28bead86c4b6f756e2f"
server_key_version: "encryptionkey@0"
I20260924 16:11:12.430420 2411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:11:12.432898 2426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:12.434144 2411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:11:12.434350 2411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6"
format_stamp: "Formatted at 2026-09-24 16:11:12 on dist-test-slave-vj6n"
server_key: "11269a71e03ec60813146acd6d762fed"
server_key_iv: "916b4fea2489f28bead86c4b6f756e2f"
server_key_version: "encryptionkey@0"
I20260924 16:11:12.434470 2411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:12.453984 2411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:12.457522 2411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:12.457743 2411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:12.465224 2411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35351
I20260924 16:11:12.465288 2478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35351 every 8 connection(s)
I20260924 16:11:12.466322 2411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:12.469316 2479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:12.470189 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2411
I20260924 16:11:12.470358 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:12.470657 13345 external_mini_cluster.cc:1428] Setting key 3b0cb05bca14ec22393e40e7475c05c7
I20260924 16:11:12.475988 2479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6: Bootstrap starting.
I20260924 16:11:12.478721 2479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:12.479503 2479 log.cc:824] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:12.479940 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:12.472094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:53103 (local address 127.0.0.1:35351)
0924 16:11:12.472515 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:12.472541 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:12.472578 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:11:12.473235 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:12.473253 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:12.473604 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:12.473885 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:12.474327 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:12.475309 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:12.476492 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:12.476959 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:12.477079 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:12.478013 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:12.478018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:12.478031 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:12.478501 (+ 470us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:12.478556 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:12.478580 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:12.478687 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:12.478946 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 16:11:12.479096 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":139,"threads_started":1}
I20260924 16:11:12.481875 2479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6: No bootstrap required, opened a new log
I20260924 16:11:12.483112 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:12.487067 2479 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } }
I20260924 16:11:12.487376 2479 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:12.487445 2479 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa669b0e2f6540dbbc9fbd8d8fb864a6, State: Initialized, Role: FOLLOWER
I20260924 16:11:12.487957 2479 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } }
I20260924 16:11:12.488133 2479 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:12.488232 2479 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:12.488353 2479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:12.489776 2479 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } }
I20260924 16:11:12.489930 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:12.485456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:43159 (local address 127.0.0.1:35351)
0924 16:11:12.485577 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:12.485581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:12.485754 (+ 173us) server_negotiation.cc:407] Connection header received
0924 16:11:12.485882 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:12.485884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:12.485922 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:12.485985 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:12.486350 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:12.486868 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:12.487972 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:12.488332 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:12.488456 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:12.489367 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:12.489370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:12.489372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:12.489384 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:12.489408 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:12.489418 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:12.489480 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:12.489681 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 16:11:12.489713 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:11:12.490288 2479 leader_election.cc:302] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa669b0e2f6540dbbc9fbd8d8fb864a6; no voters:
I20260924 16:11:12.490702 2479 leader_election.cc:288] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:12.490816 2489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:12.491147 2489 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [term 1 LEADER]: Becoming Leader. State: Replica: aa669b0e2f6540dbbc9fbd8d8fb864a6, State: Running, Role: LEADER
I20260924 16:11:12.491461 2489 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } }
I20260924 16:11:12.492050 2479 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:12.493222 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa669b0e2f6540dbbc9fbd8d8fb864a6. Latest consensus state: current_term: 1 leader_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } } }
I20260924 16:11:12.493369 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:12.493708 2490 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa669b0e2f6540dbbc9fbd8d8fb864a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35351 } } }
I20260924 16:11:12.493793 2498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:12.493829 2490 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:12.497140 2498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:12.503472 2498 catalog_manager.cc:1347] Generated new cluster ID: a0cadb5008ea4a62a934495dd73486cf
I20260924 16:11:12.503598 2498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:12.513381 2498 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:12.514442 2498 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:12.519483 2498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa669b0e2f6540dbbc9fbd8d8fb864a6: Generated new TSK 0
I20260924 16:11:12.520172 2498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:12.527232 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2411
W20260924 16:11:12.533598 2396 connection.cc:535] client connection to 127.0.0.1:35351 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35351): Connection reset by peer (error 104)
2026-09-24T16:11:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:12 dist-test-slave-vj6n krb5kdc[2510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:12 dist-test-slave-vj6n krb5kdc[2510](info): set up 2 sockets
Sep 24 16:11:12 dist-test-slave-vj6n krb5kdc[2510](info): commencing operation
krb5kdc: starting...
W20260924 16:11:13.595726 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:13 dist-test-slave-vj6n krb5kdc[2510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 16:11:13.721161 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:13.833076 2523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:13.833349 2523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:13.833392 2523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:13.837355 2523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:13.837423 2523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:13.837443 2523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:13.837460 2523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:13.837476 2523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:13.842724 2523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33685
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:13.843734 2523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:13.844717 2523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:13.853914 2528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:13.853895 2529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:13.854007 2531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:13.854315 2523 server_base.cc:1034] running on GCE node
I20260924 16:11:13.854866 2523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:13.856006 2523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:13.857213 2523 hybrid_clock.cc:648] HybridClock initialized: now 1790266273857197 us; error 27 us; skew 500 ppm
Sep 24 16:11:13 dist-test-slave-vj6n krb5kdc[2510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:13.868055 2523 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 16:11:13.869062 2523 webserver.cc:460] Webserver started at http://127.0.0.1:35371/ using document root <none> and password file <none>
I20260924 16:11:13.869645 2523 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:13.869706 2523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:13.869879 2523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:13.872103 2523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "344310c8a82f46de9efc2cc8ef640af5"
format_stamp: "Formatted at 2026-09-24 16:11:13 on dist-test-slave-vj6n"
server_key: "98140d73c54e19775a42dbea9458d90b"
server_key_iv: "168bc4167ea7ff08e193dfc1484c2a7d"
server_key_version: "encryptionkey@0"
I20260924 16:11:13.872630 2523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "344310c8a82f46de9efc2cc8ef640af5"
format_stamp: "Formatted at 2026-09-24 16:11:13 on dist-test-slave-vj6n"
server_key: "98140d73c54e19775a42dbea9458d90b"
server_key_iv: "168bc4167ea7ff08e193dfc1484c2a7d"
server_key_version: "encryptionkey@0"
I20260924 16:11:13.876507 2523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:11:13.879136 2538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:13.880465 2523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 16:11:13.880616 2523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "344310c8a82f46de9efc2cc8ef640af5"
format_stamp: "Formatted at 2026-09-24 16:11:13 on dist-test-slave-vj6n"
server_key: "98140d73c54e19775a42dbea9458d90b"
server_key_iv: "168bc4167ea7ff08e193dfc1484c2a7d"
server_key_version: "encryptionkey@0"
I20260924 16:11:13.880740 2523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:13.885600 2523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:13.888650 2523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:13.888854 2523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:13.896466 2523 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33685
I20260924 16:11:13.896485 2590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33685 every 8 connection(s)
I20260924 16:11:13.897480 2523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:13.900388 2591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:13.906126 2591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5: Bootstrap starting.
I20260924 16:11:13.907768 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2523
I20260924 16:11:13.907915 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:13.908207 13345 external_mini_cluster.cc:1428] Setting key b23e2759ef64335d7068f1c0be72f321
I20260924 16:11:13.909070 2591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:13.909974 2591 log.cc:824] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:13.912247 2591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5: No bootstrap required, opened a new log
I20260924 16:11:13.913606 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:13.909424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:46965 (local address 127.0.0.1:33685)
0924 16:11:13.909879 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:13.909894 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:13.909931 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:11:13.910844 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:13.910871 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:13.911161 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:13.911545 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:13.912364 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:13.912369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:13.912386 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:13.912399 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:13.912460 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:13.912480 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:13.912591 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:13.912718 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 16:11:13.912890 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":135,"threads_started":1}
I20260924 16:11:13.916699 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:13.916718 2591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260924 16:11:13.917027 2591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:13.917105 2591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344310c8a82f46de9efc2cc8ef640af5, State: Initialized, Role: FOLLOWER
I20260924 16:11:13.917590 2591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260924 16:11:13.917773 2591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:13.917877 2591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:13.918040 2591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:13.919443 2591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260924 16:11:13.920017 2591 leader_election.cc:302] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 344310c8a82f46de9efc2cc8ef640af5; no voters:
I20260924 16:11:13.920425 2591 leader_election.cc:288] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:13.920729 2601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:13.920989 2601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [term 1 LEADER]: Becoming Leader. State: Replica: 344310c8a82f46de9efc2cc8ef640af5, State: Running, Role: LEADER
I20260924 16:11:13.921376 2601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260924 16:11:13.921675 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:13.919545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:35227 (local address 127.0.0.1:33685)
0924 16:11:13.919671 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:13.919676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:13.920122 (+ 446us) server_negotiation.cc:407] Connection header received
0924 16:11:13.920256 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:13.920259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:13.920299 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:13.920370 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:13.921222 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:13.921226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:13.921228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:13.921241 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:13.921269 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:13.921282 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:13.921337 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:13.921515 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 16:11:13.921560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 16:11:13.921985 2591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:13.923190 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "344310c8a82f46de9efc2cc8ef640af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } } }
I20260924 16:11:13.923370 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:13.923321 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 344310c8a82f46de9efc2cc8ef640af5. Latest consensus state: current_term: 1 leader_uuid: "344310c8a82f46de9efc2cc8ef640af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "344310c8a82f46de9efc2cc8ef640af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } } }
I20260924 16:11:13.923502 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:13.923839 2610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:13.927023 2610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:13.932340 2610 catalog_manager.cc:1347] Generated new cluster ID: b119c5b4105c4421a6d7fb7ce3361a67
I20260924 16:11:13.932420 2610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:13.957942 2610 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:13.958438 2610 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:13.970461 2610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 344310c8a82f46de9efc2cc8ef640af5: Generated new TSK 0
I20260924 16:11:13.971483 2610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:13.982676 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2523
2026-09-24T16:11:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:14 dist-test-slave-vj6n krb5kdc[2622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:14 dist-test-slave-vj6n krb5kdc[2622](info): set up 2 sockets
Sep 24 16:11:14 dist-test-slave-vj6n krb5kdc[2622](info): commencing operation
krb5kdc: starting...
W20260924 16:11:15.031381 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:15.158277 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41421
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:33483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:15.273913 2635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:15.274279 2635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:15.274371 2635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:15.278606 2635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:15.278673 2635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:15.278693 2635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:15.278743 2635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:15.278774 2635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:15.284451 2635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:33483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:15.285619 2635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:15.286842 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:15.295882 2640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:15.295917 2641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:15.295949 2643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:15.296226 2635 server_base.cc:1034] running on GCE node
I20260924 16:11:15.296854 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:15.297976 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:15.299134 2635 hybrid_clock.cc:648] HybridClock initialized: now 1790266275299093 us; error 58 us; skew 500 ppm
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266275, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:15.301910 2635 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:15.302914 2635 webserver.cc:460] Webserver started at http://127.13.8.126:40157/ using document root <none> and password file <none>
I20260924 16:11:15.303467 2635 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:15.303526 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:15.303694 2635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:15.305837 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "3688948b728a42b8ba54a112d46f71e5"
format_stamp: "Formatted at 2026-09-24 16:11:15 on dist-test-slave-vj6n"
server_key: "3c53e73a0f16d21035fa844136e1943e"
server_key_iv: "e5e9ac72ddf825f25a13b71822d27fca"
server_key_version: "encryptionkey@0"
I20260924 16:11:15.306449 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "3688948b728a42b8ba54a112d46f71e5"
format_stamp: "Formatted at 2026-09-24 16:11:15 on dist-test-slave-vj6n"
server_key: "3c53e73a0f16d21035fa844136e1943e"
server_key_iv: "e5e9ac72ddf825f25a13b71822d27fca"
server_key_version: "encryptionkey@0"
I20260924 16:11:15.310598 2635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:11:15.313248 2650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:15.314692 2635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:15.314855 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "3688948b728a42b8ba54a112d46f71e5"
format_stamp: "Formatted at 2026-09-24 16:11:15 on dist-test-slave-vj6n"
server_key: "3c53e73a0f16d21035fa844136e1943e"
server_key_iv: "e5e9ac72ddf825f25a13b71822d27fca"
server_key_version: "encryptionkey@0"
I20260924 16:11:15.314985 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:15.345255 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:15.348690 2635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:15.348919 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:15.356477 2635 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41421
I20260924 16:11:15.356506 2703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41421 every 8 connection(s)
I20260924 16:11:15.357551 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:15.360628 2704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:15.365245 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2635
I20260924 16:11:15.365383 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:15.365665 13345 external_mini_cluster.cc:1428] Setting key 1679cd10253cf83a1fd0ae6b1ccbbe14
I20260924 16:11:15.367241 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: Bootstrap starting.
I20260924 16:11:15.369678 2704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:15.370553 2704 log.cc:824] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:15.372531 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: No bootstrap required, opened a new log
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:15.376663 2704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } }
I20260924 16:11:15.376976 2704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:15.377084 2704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3688948b728a42b8ba54a112d46f71e5, State: Initialized, Role: FOLLOWER
I20260924 16:11:15.377606 2704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } }
I20260924 16:11:15.377789 2704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:15.377878 2704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:15.378002 2704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:15.378998 2704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } }
I20260924 16:11:15.379370 2704 leader_election.cc:302] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3688948b728a42b8ba54a112d46f71e5; no voters:
I20260924 16:11:15.379711 2704 leader_election.cc:288] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:15.379827 2709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:15.380069 2709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3688948b728a42b8ba54a112d46f71e5, State: Running, Role: LEADER
I20260924 16:11:15.380482 2709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } }
I20260924 16:11:15.381250 2704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:15.382730 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:15.367310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50241 (local address 127.13.8.126:41421)
0924 16:11:15.367834 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:15.367852 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:15.367892 (+ 40us) server_negotiation.cc:407] Connection header received
0924 16:11:15.368626 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:15.368644 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:15.369018 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:15.369386 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:15.370366 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:15.371457 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:15.372449 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:15.373003 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:15.373199 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:15.376119 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:15.376124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:15.376139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:15.376171 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:15.378422 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:15.379047 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:15.379054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:15.379060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:15.379251 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:15.379650 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:15.379654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:15.379656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:15.380298 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:15.380627 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:15.381288 (+ 661us) server_negotiation.cc:299] Negotiation successful
0924 16:11:15.381534 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":163,"threads_started":1}
I20260924 16:11:15.384223 2711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3688948b728a42b8ba54a112d46f71e5. Latest consensus state: current_term: 1 leader_uuid: "3688948b728a42b8ba54a112d46f71e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } } }
I20260924 16:11:15.384665 2711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:15.384352 2710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3688948b728a42b8ba54a112d46f71e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3688948b728a42b8ba54a112d46f71e5" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41421 } } }
I20260924 16:11:15.384966 2710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:15.386641 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260924 16:11:15.387017 2723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:15.387102 2723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:15.388599 2725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:15.389696 2725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:15.396546 2725 catalog_manager.cc:1347] Generated new cluster ID: 647f9c1dc21a4a3daf45c5c8ad285535
I20260924 16:11:15.396642 2725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:15.399197 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:15.388919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47361 (local address 127.13.8.126:41421)
0924 16:11:15.389037 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:15.389042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:15.389339 (+ 297us) server_negotiation.cc:407] Connection header received
0924 16:11:15.389452 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:15.389456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:15.389508 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:15.389605 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:15.390527 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:15.391293 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:15.392539 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:15.392874 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:15.392999 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:15.394890 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:15.394894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:15.394897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:15.394925 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:15.396994 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:15.397558 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:15.397562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:15.397564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:15.397660 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:15.398051 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:15.398055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:15.398057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:15.398348 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:15.398522 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:15.398902 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 16:11:15.399009 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 16:11:15.404217 2725 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:15.405118 2725 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:15.415189 2725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3688948b728a42b8ba54a112d46f71e5: Generated new TSK 0
I20260924 16:11:15.416312 2725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:15.428516 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2635
W20260924 16:11:15.434966 2620 connection.cc:535] client connection to 127.13.8.126:41421 recv error: Network error: failed to read from TLS socket (remote: 127.13.8.126:41421): Connection reset by peer (error 104)
2026-09-24T16:11:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2735](info): set up 2 sockets
Sep 24 16:11:15 dist-test-slave-vj6n krb5kdc[2735](info): commencing operation
krb5kdc: starting...
W20260924 16:11:16.475612 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:16 dist-test-slave-vj6n krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:16.613643 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:37719
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:16.730274 2748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:16.730618 2748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:16.730719 2748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:16.734771 2748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:16.734875 2748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:16.734915 2748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:16.734977 2748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:16.735015 2748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:16.740701 2748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:37719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:37719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:16.741957 2748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:16.743119 2748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:16.752461 2754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:16.752521 2753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:16.752518 2748 server_base.cc:1034] running on GCE node
W20260924 16:11:16.752461 2756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:16.753314 2748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:16.754583 2748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:16.755748 2748 hybrid_clock.cc:648] HybridClock initialized: now 1790266276755727 us; error 38 us; skew 500 ppm
Sep 24 16:11:16 dist-test-slave-vj6n krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266276, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:16.758693 2748 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:16.759651 2748 webserver.cc:460] Webserver started at http://127.13.8.126:37279/ using document root <none> and password file <none>
I20260924 16:11:16.760283 2748 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:16.760344 2748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:16.760581 2748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:16.762957 2748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "134b14b40ff14b28ad7c87f684538be3"
format_stamp: "Formatted at 2026-09-24 16:11:16 on dist-test-slave-vj6n"
server_key: "ee72198206c437808ae1c35347bdb722"
server_key_iv: "4690b032827db496682fd00da322a3ff"
server_key_version: "encryptionkey@0"
I20260924 16:11:16.763556 2748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "134b14b40ff14b28ad7c87f684538be3"
format_stamp: "Formatted at 2026-09-24 16:11:16 on dist-test-slave-vj6n"
server_key: "ee72198206c437808ae1c35347bdb722"
server_key_iv: "4690b032827db496682fd00da322a3ff"
server_key_version: "encryptionkey@0"
I20260924 16:11:16.767642 2748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:11:16.770373 2763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:16.771688 2748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:16.771855 2748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "134b14b40ff14b28ad7c87f684538be3"
format_stamp: "Formatted at 2026-09-24 16:11:16 on dist-test-slave-vj6n"
server_key: "ee72198206c437808ae1c35347bdb722"
server_key_iv: "4690b032827db496682fd00da322a3ff"
server_key_version: "encryptionkey@0"
I20260924 16:11:16.771978 2748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:16.813810 2748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:16.817256 2748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:16.817477 2748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:16.824981 2748 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:37719
I20260924 16:11:16.825016 2816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:37719 every 8 connection(s)
I20260924 16:11:16.825980 2748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:16.828900 2817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:16.831383 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2748
I20260924 16:11:16.831519 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:16.831794 13345 external_mini_cluster.cc:1428] Setting key c45833a82cee1daaa0cbe9796d979d08
I20260924 16:11:16.835353 2817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: Bootstrap starting.
I20260924 16:11:16.837762 2817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:16.838624 2817 log.cc:824] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:16.841396 2817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: No bootstrap required, opened a new log
I20260924 16:11:16.841934 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:16.833312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44433 (local address 127.13.8.126:37719)
0924 16:11:16.833779 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:16.833794 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:16.833832 (+ 38us) server_negotiation.cc:407] Connection header received
0924 16:11:16.834567 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:16.834589 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:16.834964 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:16.835337 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:16.835761 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:16.836758 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:16.837688 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:16.838283 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:16.838446 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:16.839499 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:16.839504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:16.839518 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:16.840366 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:16.840437 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:16.840468 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:16.840635 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:16.840822 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 16:11:16.840964 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":142,"threads_started":1}
I20260924 16:11:16.845176 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:16.845870 2817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } }
I20260924 16:11:16.846179 2817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:16.846256 2817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134b14b40ff14b28ad7c87f684538be3, State: Initialized, Role: FOLLOWER
I20260924 16:11:16.846849 2817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } }
I20260924 16:11:16.847033 2817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:16.847128 2817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:16.847268 2817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:16.848639 2817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } }
I20260924 16:11:16.849015 2817 leader_election.cc:302] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 134b14b40ff14b28ad7c87f684538be3; no voters:
I20260924 16:11:16.849402 2817 leader_election.cc:288] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:16.849474 2827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:16.849704 2827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [term 1 LEADER]: Becoming Leader. State: Replica: 134b14b40ff14b28ad7c87f684538be3, State: Running, Role: LEADER
I20260924 16:11:16.850071 2827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } }
I20260924 16:11:16.850869 2817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:16.851406 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "134b14b40ff14b28ad7c87f684538be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } } }
I20260924 16:11:16.852264 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:16.852303 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:16.847591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45623 (local address 127.13.8.126:37719)
0924 16:11:16.847734 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:16.847738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:16.847925 (+ 187us) server_negotiation.cc:407] Connection header received
0924 16:11:16.848046 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:16.848048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:16.848089 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:16.848153 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:16.848483 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:16.849099 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:16.850253 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:16.850509 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:16.850627 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:16.851571 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:16.851576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:16.851578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:16.851593 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:16.851623 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:16.851636 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:16.851730 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:16.852084 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 16:11:16.852120 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 16:11:16.852139 2828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134b14b40ff14b28ad7c87f684538be3. Latest consensus state: current_term: 1 leader_uuid: "134b14b40ff14b28ad7c87f684538be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b14b40ff14b28ad7c87f684538be3" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37719 } } }
I20260924 16:11:16.852931 2828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3 [sys.catalog]: This master's current role is: LEADER
W20260924 16:11:16.856724 2842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:16.856824 2842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:16.856954 2841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:16.857748 2841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:16.863688 2841 catalog_manager.cc:1347] Generated new cluster ID: 3afbe54b129347d1b487b2d49356a414
I20260924 16:11:16.863772 2841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:16.880144 2841 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:16.881218 2841 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:16.889672 2841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 134b14b40ff14b28ad7c87f684538be3: Generated new TSK 0
I20260924 16:11:16.890486 2841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:16.899573 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2748
W20260924 16:11:16.905472 2733 connection.cc:535] client connection to 127.13.8.126:37719 recv error: Network error: failed to read from TLS socket (remote: 127.13.8.126:37719): Connection reset by peer (error 104)
2026-09-24T16:11:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:16 dist-test-slave-vj6n krb5kdc[2848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:16 dist-test-slave-vj6n krb5kdc[2848](info): set up 2 sockets
Sep 24 16:11:16 dist-test-slave-vj6n krb5kdc[2848](info): commencing operation
krb5kdc: starting...
W20260924 16:11:17.944383 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:18 dist-test-slave-vj6n krb5kdc[2848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:18.069545 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:42103
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:18.183416 2861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:18.183754 2861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:18.183840 2861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:18.187901 2861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:18.188000 2861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:18.188040 2861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:18.188099 2861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:18.188136 2861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:18.193562 2861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:42103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:18.194780 2861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:18.195861 2861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:18.204937 2869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:18.204960 2861 server_base.cc:1034] running on GCE node
W20260924 16:11:18.204991 2866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:18.204946 2867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:18.205808 2861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:18.206996 2861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:18.208163 2861 hybrid_clock.cc:648] HybridClock initialized: now 1790266278208129 us; error 45 us; skew 500 ppm
Sep 24 16:11:18 dist-test-slave-vj6n krb5kdc[2848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266278, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:18.211123 2861 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:18.212106 2861 webserver.cc:460] Webserver started at http://127.13.8.126:41457/ using document root <none> and password file <none>
I20260924 16:11:18.212730 2861 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:18.212831 2861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:18.213076 2861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:18.215269 2861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70"
format_stamp: "Formatted at 2026-09-24 16:11:18 on dist-test-slave-vj6n"
server_key: "7e12f6fbae1b9cda1d1a99ef4f239678"
server_key_iv: "a36c886d8ae22fca25cefde1ce1e6e4a"
server_key_version: "encryptionkey@0"
I20260924 16:11:18.215857 2861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70"
format_stamp: "Formatted at 2026-09-24 16:11:18 on dist-test-slave-vj6n"
server_key: "7e12f6fbae1b9cda1d1a99ef4f239678"
server_key_iv: "a36c886d8ae22fca25cefde1ce1e6e4a"
server_key_version: "encryptionkey@0"
I20260924 16:11:18.219776 2861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 16:11:18.222410 2876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:18.223695 2861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:18.223847 2861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70"
format_stamp: "Formatted at 2026-09-24 16:11:18 on dist-test-slave-vj6n"
server_key: "7e12f6fbae1b9cda1d1a99ef4f239678"
server_key_iv: "a36c886d8ae22fca25cefde1ce1e6e4a"
server_key_version: "encryptionkey@0"
I20260924 16:11:18.223964 2861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:18.228816 2861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:18.232192 2861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:18.232398 2861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:18.239892 2861 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:42103
I20260924 16:11:18.239952 2928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:42103 every 8 connection(s)
I20260924 16:11:18.240906 2861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:18.244711 2929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:18.246049 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2861
I20260924 16:11:18.246176 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:18.246526 13345 external_mini_cluster.cc:1428] Setting key 5438dcd18431b6f03730b3c56509bc52
I20260924 16:11:18.253155 2929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70: Bootstrap starting.
I20260924 16:11:18.255909 2929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:18.256732 2929 log.cc:824] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:18.258993 2929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70: No bootstrap required, opened a new log
I20260924 16:11:18.261574 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:18.257297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48485 (local address 127.13.8.126:42103)
0924 16:11:18.257753 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:18.257772 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:18.257828 (+ 56us) server_negotiation.cc:407] Connection header received
0924 16:11:18.258711 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:18.258733 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:18.259015 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:18.259363 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:18.260308 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:18.260312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:18.260323 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:18.260334 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:18.260396 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:18.260415 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:18.260504 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:18.260643 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 16:11:18.260823 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
I20260924 16:11:18.263209 2929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } }
I20260924 16:11:18.263477 2929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:18.263566 2929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1057e3ba34d35a7d4cd98ea93ce70, State: Initialized, Role: FOLLOWER
I20260924 16:11:18.264074 2929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } }
I20260924 16:11:18.264271 2929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:18.264357 2929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:18.264460 2929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:18.264918 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:18.265532 2929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } }
I20260924 16:11:18.265893 2929 leader_election.cc:302] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bc1057e3ba34d35a7d4cd98ea93ce70; no voters:
I20260924 16:11:18.266211 2929 leader_election.cc:288] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:18.266373 2934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:18.266661 2934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1057e3ba34d35a7d4cd98ea93ce70, State: Running, Role: LEADER
I20260924 16:11:18.267042 2934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } }
I20260924 16:11:18.267652 2929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:18.268914 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } } }
I20260924 16:11:18.269047 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:18.269332 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bc1057e3ba34d35a7d4cd98ea93ce70. Latest consensus state: current_term: 1 leader_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc1057e3ba34d35a7d4cd98ea93ce70" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 42103 } } }
I20260924 16:11:18.269465 2948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:18.269490 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:18.270661 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:18.267306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44175 (local address 127.13.8.126:42103)
0924 16:11:18.267431 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:18.267436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:18.267857 (+ 421us) server_negotiation.cc:407] Connection header received
0924 16:11:18.267994 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:18.267998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:18.268045 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:18.268123 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:18.270179 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:18.270184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:18.270187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:18.270208 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:18.270263 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:18.270277 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:18.270364 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:18.270502 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 16:11:18.270544 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:18.273197 2948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:18.278761 2948 catalog_manager.cc:1347] Generated new cluster ID: 559a15649b484f988e28f77a7e4257f2
I20260924 16:11:18.278827 2948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:18.309785 2948 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:18.310143 2948 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:18.316285 2948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bc1057e3ba34d35a7d4cd98ea93ce70: Generated new TSK 0
I20260924 16:11:18.317179 2948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:18.326929 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2861
2026-09-24T16:11:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:18 dist-test-slave-vj6n krb5kdc[2960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:18 dist-test-slave-vj6n krb5kdc[2960](info): set up 2 sockets
Sep 24 16:11:18 dist-test-slave-vj6n krb5kdc[2960](info): commencing operation
krb5kdc: starting...
W20260924 16:11:19.367594 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[2960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:19.487772 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41055
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:41055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:19.600126 2973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:19.600395 2973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:19.600477 2973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:19.604496 2973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:19.604553 2973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:19.604573 2973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:19.604589 2973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:19.604655 2973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:19.609987 2973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:41055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:41055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:19.611119 2973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:19.612229 2973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:19.621450 2979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:19.621539 2978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:19.621752 2973 server_base.cc:1034] running on GCE node
W20260924 16:11:19.621546 2981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:19.622388 2973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:19.623498 2973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:19.624665 2973 hybrid_clock.cc:648] HybridClock initialized: now 1790266279624632 us; error 42 us; skew 500 ppm
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[2960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266279, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:19.631925 2973 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:19.632911 2973 webserver.cc:460] Webserver started at http://127.13.8.126:41613/ using document root <none> and password file <none>
I20260924 16:11:19.633461 2973 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:19.633520 2973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:19.633690 2973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:19.635780 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "fec46f4b207d4875b98dcff23172c840"
format_stamp: "Formatted at 2026-09-24 16:11:19 on dist-test-slave-vj6n"
server_key: "3c64eb563abca9ba819b8604de91959a"
server_key_iv: "46fa68319ff46b4be20eff5e46d7c09a"
server_key_version: "encryptionkey@0"
I20260924 16:11:19.636360 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "fec46f4b207d4875b98dcff23172c840"
format_stamp: "Formatted at 2026-09-24 16:11:19 on dist-test-slave-vj6n"
server_key: "3c64eb563abca9ba819b8604de91959a"
server_key_iv: "46fa68319ff46b4be20eff5e46d7c09a"
server_key_version: "encryptionkey@0"
I20260924 16:11:19.640165 2973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 16:11:19.642544 2988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:19.643800 2973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 16:11:19.643942 2973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "fec46f4b207d4875b98dcff23172c840"
format_stamp: "Formatted at 2026-09-24 16:11:19 on dist-test-slave-vj6n"
server_key: "3c64eb563abca9ba819b8604de91959a"
server_key_iv: "46fa68319ff46b4be20eff5e46d7c09a"
server_key_version: "encryptionkey@0"
I20260924 16:11:19.644054 2973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:19.666875 2973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:19.670356 2973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:19.670617 2973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:19.678081 2973 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:41055
I20260924 16:11:19.678102 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:41055 every 8 connection(s)
I20260924 16:11:19.679164 2973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:19.682016 3041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:19.684581 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 2973
I20260924 16:11:19.684737 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:19.685019 13345 external_mini_cluster.cc:1428] Setting key 164ec17c10968390abb1ac2ef4bbbfb0
I20260924 16:11:19.688247 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840: Bootstrap starting.
I20260924 16:11:19.690596 3041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:19.691360 3041 log.cc:824] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:19.693274 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840: No bootstrap required, opened a new log
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[2960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:19.696380 3041 raft_consensus.cc:348] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } }
I20260924 16:11:19.696630 3041 raft_consensus.cc:374] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:19.696712 3041 raft_consensus.cc:729] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec46f4b207d4875b98dcff23172c840, State: Initialized, Role: FOLLOWER
I20260924 16:11:19.697276 3041 consensus_queue.cc:260] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } }
I20260924 16:11:19.697484 3041 raft_consensus.cc:388] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:19.697587 3041 raft_consensus.cc:482] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:19.697688 3041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:19.698921 3041 raft_consensus.cc:504] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } }
I20260924 16:11:19.699299 3041 leader_election.cc:302] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fec46f4b207d4875b98dcff23172c840; no voters:
I20260924 16:11:19.699625 3041 leader_election.cc:288] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:19.699746 3046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:19.699985 3046 raft_consensus.cc:686] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [term 1 LEADER]: Becoming Leader. State: Replica: fec46f4b207d4875b98dcff23172c840, State: Running, Role: LEADER
I20260924 16:11:19.700146 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:19.686499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57017 (local address 127.13.8.126:41055)
0924 16:11:19.686960 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:19.686977 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:19.687014 (+ 37us) server_negotiation.cc:407] Connection header received
0924 16:11:19.687647 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:19.687663 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:19.688026 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:19.688379 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:19.689209 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:19.690213 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:19.691205 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:19.691751 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:19.691919 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:19.694380 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:19.694385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:19.694399 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:19.694427 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:19.696777 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:19.697307 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:19.697312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:19.697317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:19.697426 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:19.697796 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:19.697799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:19.697801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:19.698360 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:19.698573 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:19.698811 (+ 238us) server_negotiation.cc:299] Negotiation successful
0924 16:11:19.699135 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":134,"threads_started":1}
I20260924 16:11:19.700476 3046 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } }
I20260924 16:11:19.700922 3041 sys_catalog.cc:564] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:19.702415 3047 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fec46f4b207d4875b98dcff23172c840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } } }
I20260924 16:11:19.702534 3047 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:19.702986 3057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:19.702214 3048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec46f4b207d4875b98dcff23172c840. Latest consensus state: current_term: 1 leader_uuid: "fec46f4b207d4875b98dcff23172c840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec46f4b207d4875b98dcff23172c840" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 41055 } } }
I20260924 16:11:19.703817 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:19.703809 3048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:19.706367 3057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:19.712911 3057 catalog_manager.cc:1347] Generated new cluster ID: bcd427627e314dea85d51782516dff5c
I20260924 16:11:19.713023 3057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:19.715334 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:19.705963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55499 (local address 127.13.8.126:41055)
0924 16:11:19.706077 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:19.706083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:19.706323 (+ 240us) server_negotiation.cc:407] Connection header received
0924 16:11:19.706464 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:19.706467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:19.706517 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:19.706606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:19.707499 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:19.708319 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:19.709420 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:19.709833 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:19.709946 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:19.711658 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:19.711662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:19.711665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:19.711700 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:19.713520 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:19.714003 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:19.714007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:19.714009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:19.714101 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:19.714426 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:19.714429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:19.714431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:19.714656 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:19.714805 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:19.715049 (+ 244us) server_negotiation.cc:299] Negotiation successful
0924 16:11:19.715174 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 16:11:19.735394 3057 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:19.736447 3057 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:19.752466 3057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fec46f4b207d4875b98dcff23172c840: Generated new TSK 0
I20260924 16:11:19.753309 3057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:19.762754 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 2973
2026-09-24T16:11:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[3072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[3072](info): set up 2 sockets
Sep 24 16:11:19 dist-test-slave-vj6n krb5kdc[3072](info): commencing operation
krb5kdc: starting...
W20260924 16:11:20.807572 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:20 dist-test-slave-vj6n krb5kdc[3072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:20.929910 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36435
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:36435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:21.045430 3085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:21.045742 3085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:21.045828 3085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:21.049842 3085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:21.049940 3085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:21.049980 3085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:21.050016 3085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:21.050045 3085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:21.055342 3085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:36435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:21.056483 3085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:21.057520 3085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:21.066300 3093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:21.066351 3090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:21.066525 3085 server_base.cc:1034] running on GCE node
W20260924 16:11:21.066530 3091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:21.067199 3085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:21.068374 3085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:21.069556 3085 hybrid_clock.cc:648] HybridClock initialized: now 1790266281069526 us; error 42 us; skew 500 ppm
Sep 24 16:11:21 dist-test-slave-vj6n krb5kdc[3072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266281, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:21.072480 3085 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:21.073388 3085 webserver.cc:460] Webserver started at http://127.13.8.126:34439/ using document root <none> and password file <none>
I20260924 16:11:21.073968 3085 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:21.074056 3085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:21.074365 3085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:21.076495 3085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "075afae6cc3a4be9bee77d3c5279f247"
format_stamp: "Formatted at 2026-09-24 16:11:21 on dist-test-slave-vj6n"
server_key: "2eebfe9972569b896696628193b72713"
server_key_iv: "4190dd071300c8e236b4129e6b0d82ec"
server_key_version: "encryptionkey@0"
I20260924 16:11:21.077088 3085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "075afae6cc3a4be9bee77d3c5279f247"
format_stamp: "Formatted at 2026-09-24 16:11:21 on dist-test-slave-vj6n"
server_key: "2eebfe9972569b896696628193b72713"
server_key_iv: "4190dd071300c8e236b4129e6b0d82ec"
server_key_version: "encryptionkey@0"
I20260924 16:11:21.081183 3085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:11:21.084071 3100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:21.085510 3085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 16:11:21.085685 3085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "075afae6cc3a4be9bee77d3c5279f247"
format_stamp: "Formatted at 2026-09-24 16:11:21 on dist-test-slave-vj6n"
server_key: "2eebfe9972569b896696628193b72713"
server_key_iv: "4190dd071300c8e236b4129e6b0d82ec"
server_key_version: "encryptionkey@0"
I20260924 16:11:21.085830 3085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:21.101529 3085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:21.104643 3085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:21.104859 3085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:21.112030 3085 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:36435
I20260924 16:11:21.112073 3152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:36435 every 8 connection(s)
I20260924 16:11:21.112987 3085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:21.115974 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:21.116257 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3085
I20260924 16:11:21.116410 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:21.116714 13345 external_mini_cluster.cc:1428] Setting key 04c1d4b3587cb1a34cbc48abb99d0d39
I20260924 16:11:21.124092 3153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247: Bootstrap starting.
I20260924 16:11:21.127290 3153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:21.127604 3156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:21.118272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46755 (local address 127.13.8.126:36435)
0924 16:11:21.118751 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:21.118773 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:21.118823 (+ 50us) server_negotiation.cc:407] Connection header received
0924 16:11:21.119712 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:21.119739 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:21.120227 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:21.120653 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:21.121098 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:21.122715 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:21.123615 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:21.124320 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:21.124498 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:21.125402 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:21.125408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:21.125427 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:21.125972 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:21.126041 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:21.126086 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:21.126191 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:21.126359 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 16:11:21.126574 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":109,"threads_started":1}
I20260924 16:11:21.128211 3153 log.cc:824] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:21.130635 3153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247: No bootstrap required, opened a new log
I20260924 16:11:21.130666 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:21.134531 3153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } }
I20260924 16:11:21.134806 3153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:21.134889 3153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075afae6cc3a4be9bee77d3c5279f247, State: Initialized, Role: FOLLOWER
I20260924 16:11:21.135382 3153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } }
I20260924 16:11:21.135555 3153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:21.135643 3153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:21.135767 3153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:21.136832 3156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:21.132596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42145 (local address 127.13.8.126:36435)
0924 16:11:21.132755 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:21.132761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:21.132963 (+ 202us) server_negotiation.cc:407] Connection header received
0924 16:11:21.133112 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:21.133115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:21.133165 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:21.133257 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:21.133523 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:21.134041 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:21.134993 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:21.135253 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:21.135373 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:21.136239 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:21.136293 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:21.136304 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:21.136331 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:21.136371 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:21.136390 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:21.136514 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:21.136613 (+ 99us) server_negotiation.cc:299] Negotiation successful
0924 16:11:21.136670 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 16:11:21.136818 3153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } }
I20260924 16:11:21.137253 3153 leader_election.cc:302] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 075afae6cc3a4be9bee77d3c5279f247; no voters:
I20260924 16:11:21.137614 3153 leader_election.cc:288] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:21.137686 3163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:21.137880 3163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [term 1 LEADER]: Becoming Leader. State: Replica: 075afae6cc3a4be9bee77d3c5279f247, State: Running, Role: LEADER
I20260924 16:11:21.138185 3163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } }
I20260924 16:11:21.139019 3153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:21.139430 3165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 075afae6cc3a4be9bee77d3c5279f247. Latest consensus state: current_term: 1 leader_uuid: "075afae6cc3a4be9bee77d3c5279f247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } } }
I20260924 16:11:21.139605 3165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:21.139854 3164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "075afae6cc3a4be9bee77d3c5279f247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075afae6cc3a4be9bee77d3c5279f247" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36435 } } }
I20260924 16:11:21.139997 3164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:21.140149 3168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:21.143821 3168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:21.150315 3168 catalog_manager.cc:1347] Generated new cluster ID: 0620ec2d7be4441ba5de4b6f27ffd2fb
I20260924 16:11:21.150403 3168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:21.171665 3168 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:21.173007 3168 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:21.179119 3168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 075afae6cc3a4be9bee77d3c5279f247: Generated new TSK 0
I20260924 16:11:21.179962 3168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:21.190017 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3085
2026-09-24T16:11:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:21 dist-test-slave-vj6n krb5kdc[3184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:21 dist-test-slave-vj6n krb5kdc[3184](info): set up 2 sockets
Sep 24 16:11:21 dist-test-slave-vj6n krb5kdc[3184](info): commencing operation
krb5kdc: starting...
W20260924 16:11:22.235638 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:22 dist-test-slave-vj6n krb5kdc[3184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:22.362414 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36947
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:36947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:22.476214 3197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:22.476521 3197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:22.476606 3197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:22.480778 3197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:22.480844 3197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:22.480863 3197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:22.480916 3197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:22.480952 3197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:22.487076 3197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:36947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:36947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:22.488423 3197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:22.489533 3197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:22.498309 3202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:22.498445 3203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:22.498306 3205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:22.498382 3197 server_base.cc:1034] running on GCE node
I20260924 16:11:22.499101 3197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:22.500162 3197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:22.501348 3197 hybrid_clock.cc:648] HybridClock initialized: now 1790266282501333 us; error 25 us; skew 500 ppm
Sep 24 16:11:22 dist-test-slave-vj6n krb5kdc[3184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266282, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:22.504029 3197 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:22.504896 3197 webserver.cc:460] Webserver started at http://127.13.8.126:41005/ using document root <none> and password file <none>
I20260924 16:11:22.505455 3197 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:22.505514 3197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:22.505759 3197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:22.507886 3197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "877bfe2189a1440b869f2353aad619e8"
format_stamp: "Formatted at 2026-09-24 16:11:22 on dist-test-slave-vj6n"
server_key: "02ab4101826e4a7646c953ee791c46dd"
server_key_iv: "bf6bfa018d784ab557273366708cef90"
server_key_version: "encryptionkey@0"
I20260924 16:11:22.508412 3197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "877bfe2189a1440b869f2353aad619e8"
format_stamp: "Formatted at 2026-09-24 16:11:22 on dist-test-slave-vj6n"
server_key: "02ab4101826e4a7646c953ee791c46dd"
server_key_iv: "bf6bfa018d784ab557273366708cef90"
server_key_version: "encryptionkey@0"
I20260924 16:11:22.512182 3197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260924 16:11:22.514649 3212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:22.515854 3197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:11:22.515982 3197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "877bfe2189a1440b869f2353aad619e8"
format_stamp: "Formatted at 2026-09-24 16:11:22 on dist-test-slave-vj6n"
server_key: "02ab4101826e4a7646c953ee791c46dd"
server_key_iv: "bf6bfa018d784ab557273366708cef90"
server_key_version: "encryptionkey@0"
I20260924 16:11:22.516081 3197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:22.520622 3197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:22.523602 3197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:22.523782 3197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:22.530721 3197 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:36947
I20260924 16:11:22.530778 3265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:36947 every 8 connection(s)
I20260924 16:11:22.531688 3197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:22.534554 3266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:22.538977 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3197
I20260924 16:11:22.539129 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:22.539386 13345 external_mini_cluster.cc:1428] Setting key 28816b2ba844605c6ce379c453366cf7
I20260924 16:11:22.540578 3266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8: Bootstrap starting.
I20260924 16:11:22.543452 3266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:22.544441 3266 log.cc:824] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:22.545310 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:22.540852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51111 (local address 127.13.8.126:36947)
0924 16:11:22.541298 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:22.541316 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:22.541358 (+ 42us) server_negotiation.cc:407] Connection header received
0924 16:11:22.542271 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:22.542295 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:22.542662 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:22.543027 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:22.543792 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:22.543800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:22.543816 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:22.543827 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:22.543891 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:22.543917 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:22.544029 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:22.544263 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 16:11:22.544428 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":124,"threads_started":1}
I20260924 16:11:22.546625 3266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8: No bootstrap required, opened a new log
I20260924 16:11:22.548189 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:22.550966 3266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } }
I20260924 16:11:22.551271 3266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 16:11:22.551357 3266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877bfe2189a1440b869f2353aad619e8, State: Initialized, Role: FOLLOWER
I20260924 16:11:22.551839 3266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } }
I20260924 16:11:22.551959 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:22.550202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41037 (local address 127.13.8.126:36947)
0924 16:11:22.550358 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:22.550362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:22.550553 (+ 191us) server_negotiation.cc:407] Connection header received
0924 16:11:22.550671 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:22.550673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:22.550713 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:22.550776 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:22.551568 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:22.551571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:22.551574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 16:11:22.551586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:22.551617 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 16:11:22.551628 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:22.551714 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:22.551803 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 16:11:22.551843 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 16:11:22.552017 3266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 16:11:22.552130 3266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 16:11:22.552251 3266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:22.553313 3266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } }
I20260924 16:11:22.553664 3266 leader_election.cc:302] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 877bfe2189a1440b869f2353aad619e8; no voters:
I20260924 16:11:22.553959 3266 leader_election.cc:288] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 16:11:22.554163 3276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:22.554412 3276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [term 1 LEADER]: Becoming Leader. State: Replica: 877bfe2189a1440b869f2353aad619e8, State: Running, Role: LEADER
I20260924 16:11:22.554798 3276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } }
I20260924 16:11:22.555480 3266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:22.556602 3278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877bfe2189a1440b869f2353aad619e8. Latest consensus state: current_term: 1 leader_uuid: "877bfe2189a1440b869f2353aad619e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } } }
I20260924 16:11:22.556638 3277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877bfe2189a1440b869f2353aad619e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bfe2189a1440b869f2353aad619e8" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 36947 } } }
I20260924 16:11:22.556772 3277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:22.556770 3278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8 [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:22.557600 3285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:22.560521 3285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:22.566152 3285 catalog_manager.cc:1347] Generated new cluster ID: 66dd0f7aed40431caefb3078185379de
I20260924 16:11:22.566238 3285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:22.577442 3285 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:22.577773 3285 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:22.583958 3285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877bfe2189a1440b869f2353aad619e8: Generated new TSK 0
I20260924 16:11:22.584812 3285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 16:11:22.595633 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3197
2026-09-24T16:11:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1405 ms)
[----------] 18 tests from AuthTokenIssuingTest (25942 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:22 dist-test-slave-vj6n krb5kdc[3297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:22 dist-test-slave-vj6n krb5kdc[3297](info): set up 2 sockets
Sep 24 16:11:22 dist-test-slave-vj6n krb5kdc[3297](info): commencing operation
krb5kdc: starting...
W20260924 16:11:23.647557 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:23 dist-test-slave-vj6n krb5kdc[3297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:23.771013 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:37601
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:23.885771 3310 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:23.886166 3310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:23.886274 3310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:23.886363 3310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:23.890977 3310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:23.891064 3310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:23.891104 3310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:23.891141 3310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:23.891175 3310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:23.896690 3310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:23.897830 3310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:23.898942 3310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:23.908188 3315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:23.908178 3310 server_base.cc:1034] running on GCE node
W20260924 16:11:23.908298 3318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:23.908155 3316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:23.909044 3310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:23.910254 3310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:23.911422 3310 hybrid_clock.cc:648] HybridClock initialized: now 1790266283911388 us; error 43 us; skew 500 ppm
Sep 24 16:11:23 dist-test-slave-vj6n krb5kdc[3297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:23.914212 3310 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:23.915179 3310 webserver.cc:460] Webserver started at http://127.13.8.126:42207/ using document root <none> and password file <none>
I20260924 16:11:23.915723 3310 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:23.915781 3310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:23.915948 3310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:23.918030 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "8d6d26d64b0e4101bfff381d70b02d6f"
format_stamp: "Formatted at 2026-09-24 16:11:23 on dist-test-slave-vj6n"
server_key: "d264b215dd086e1b5281ae8f3514991d"
server_key_iv: "a1cd6b6d3c9732bdb3e57374c8fb8755"
server_key_version: "encryptionkey@0"
I20260924 16:11:23.918630 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "8d6d26d64b0e4101bfff381d70b02d6f"
format_stamp: "Formatted at 2026-09-24 16:11:23 on dist-test-slave-vj6n"
server_key: "d264b215dd086e1b5281ae8f3514991d"
server_key_iv: "a1cd6b6d3c9732bdb3e57374c8fb8755"
server_key_version: "encryptionkey@0"
I20260924 16:11:23.922328 3310 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 16:11:23.924737 3325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:23.926057 3310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:11:23.926192 3310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "8d6d26d64b0e4101bfff381d70b02d6f"
format_stamp: "Formatted at 2026-09-24 16:11:23 on dist-test-slave-vj6n"
server_key: "d264b215dd086e1b5281ae8f3514991d"
server_key_iv: "a1cd6b6d3c9732bdb3e57374c8fb8755"
server_key_version: "encryptionkey@0"
I20260924 16:11:23.926350 3310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:23.949371 3310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:23.952688 3310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:23.952919 3310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:23.960767 3310 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:37601
I20260924 16:11:23.961077 3378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:37601 every 8 connection(s)
I20260924 16:11:23.961849 3310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:23.964775 3379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:23.967545 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3310
I20260924 16:11:23.967687 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:23.967976 13345 external_mini_cluster.cc:1428] Setting key f84e983ff722443178ab84a51f3eb337
I20260924 16:11:23.969327 3379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.125@KRBTEST.COM" created.
Sep 24 16:11:23 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:23.984895 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:23.972049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.13.8.126:37601)
0924 16:11:23.972554 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:23.972569 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:23.972893 (+ 324us) server_negotiation.cc:407] Connection header received
0924 16:11:23.973991 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:23.973995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:23.974172 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:23.974388 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:23.975244 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:23.976180 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:23.977239 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:23.977598 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:23.977772 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:23.980228 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:23.980234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:23.980244 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:23.980263 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:23.981790 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:23.982264 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:23.982270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:23.982272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:23.982375 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:23.982688 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:23.982693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:23.982695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:23.983064 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:23.983201 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:23.983560 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 16:11:23.983805 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":151,"threads_started":1}
I20260924 16:11:23.984895 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:23.972048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:37601 (local address 127.0.0.1:58346)
0924 16:11:23.972746 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:23.972780 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:23.973674 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:23.974385 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:23.974391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:23.974848 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:23.975096 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:23.975108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:23.976304 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:23.976307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:23.977121 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:23.977130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:23.977808 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:23.977811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:23.977913 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:23.978597 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:23.978622 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:23.980076 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:23.981905 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:23.981913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:23.981924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:23.982146 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:23.982477 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:23.982479 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:23.982481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:23.982592 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:23.983183 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:23.983189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:23.983331 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 16:11:23.983499 (+ 168us) client_negotiation.cc:240] Negotiation successful
0924 16:11:23.983883 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":98,"thread_start_us":214,"threads_started":1}
I20260924 16:11:23.987632 3379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:23.988401 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:23.988134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46832)
0924 16:11:23.988254 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:23.988333 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 16:11:23.988595 3328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
I20260924 16:11:23.989425 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:23.989216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46840)
0924 16:11:23.989319 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:23.989371 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 16:11:23.989583 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43763: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
I20260924 16:11:23.993557 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:43763
--webserver_interface=127.13.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:11:24.038869 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } attempt: 1
I20260924 16:11:24.039764 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.039455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46848)
0924 16:11:24.039602 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.039682 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:11:24.040549 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.040336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46854)
0924 16:11:24.040445 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.040493 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:11:24.040714 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43763: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
I20260924 16:11:24.108886 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } attempt: 2
W20260924 16:11:24.108914 3386 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:24.109257 3386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:24.109319 3386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:24.109391 3386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260924 16:11:24.109786 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.109491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46860)
0924 16:11:24.109623 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.109709 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 16:11:24.110544 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.110341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46862)
0924 16:11:24.110443 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.110493 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 16:11:24.110666 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:43763: Network error: Client connection negotiation failed: client connection to 127.13.8.125:43763: connect: Connection refused (error 111)
W20260924 16:11:24.113806 3386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:24.113888 3386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:24.113936 3386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:24.113973 3386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:24.114008 3386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:24.119721 3386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:43763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:24.120872 3386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:24.121954 3386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:24.130650 3391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:24.130702 3392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:24.130761 3386 server_base.cc:1034] running on GCE node
W20260924 16:11:24.130667 3394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:24.131472 3386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:24.132596 3386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:24.133788 3386 hybrid_clock.cc:648] HybridClock initialized: now 1790266284133773 us; error 24 us; skew 500 ppm
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:24.136521 3386 init.cc:377] Logged in from keytab as kudu/127.13.8.125@KRBTEST.COM (short username kudu)
I20260924 16:11:24.137362 3386 webserver.cc:460] Webserver started at http://127.13.8.125:42561/ using document root <none> and password file <none>
I20260924 16:11:24.137882 3386 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:24.137938 3386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:24.138110 3386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:24.140264 3386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data/instance:
uuid: "d583d3b64aec429e88e910adbcf58d83"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "24c8a6ae7cfd62ff3c74247fd0a6a218"
server_key_iv: "07fa5f28125cf1bee86519c8a30c03e3"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.140779 3386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal/instance:
uuid: "d583d3b64aec429e88e910adbcf58d83"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "24c8a6ae7cfd62ff3c74247fd0a6a218"
server_key_iv: "07fa5f28125cf1bee86519c8a30c03e3"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.144423 3386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 16:11:24.146811 3401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:24.148121 3386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 16:11:24.148253 3386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "d583d3b64aec429e88e910adbcf58d83"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "24c8a6ae7cfd62ff3c74247fd0a6a218"
server_key_iv: "07fa5f28125cf1bee86519c8a30c03e3"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.148361 3386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:24.177457 3386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:24.180648 3386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:24.180871 3386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:24.188181 3386 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.125:43763
I20260924 16:11:24.188470 3453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.125:43763 every 8 connection(s)
I20260924 16:11:24.189265 3386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
I20260924 16:11:24.190277 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3386
I20260924 16:11:24.190419 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-1/wal/instance
I20260924 16:11:24.190689 13345 external_mini_cluster.cc:1428] Setting key 0ee28c8456d748d5165e0e55fa8c8832
I20260924 16:11:24.192561 3454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:24.198624 3454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.8.124@KRBTEST.COM; defaulting to no policy
I20260924 16:11:24.201826 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.124@KRBTEST.COM" created.
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:24.213828 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.202349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46868)
0924 16:11:24.202721 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.202737 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.202842 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.204098 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.204104 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.204334 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.204513 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.204524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.205774 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.205779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.206388 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.206397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.207135 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.207139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.207259 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.207815 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.207841 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.209692 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.211827 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.211832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.211834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.212036 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.212369 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.212372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.212374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.212465 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.213235 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.213238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.213351 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.213459 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.213657 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260924 16:11:24.215296 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.203045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.13.8.125:43763)
0924 16:11:24.203455 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.203469 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.203511 (+ 42us) server_negotiation.cc:407] Connection header received
0924 16:11:24.203677 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.203683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.203910 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.204092 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.204656 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.205649 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.206506 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.207003 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.207191 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.209847 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.209853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.209864 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.209887 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.211716 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.212153 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.212158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.212163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.212272 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.212562 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.212566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.212568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.213103 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.213258 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.213643 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.214018 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":291,"spinlock_wait_cycles":3328,"thread_start_us":190,"threads_started":1}
I20260924 16:11:24.216076 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.201283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.13.8.126:37601)
0924 16:11:24.201428 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.201432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.202437 (+ 1005us) server_negotiation.cc:407] Connection header received
0924 16:11:24.203518 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.203525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.203593 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.203689 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.204949 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.205876 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.207535 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.207907 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.208028 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.210940 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.210945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.210950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.210984 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.213657 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.214574 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.214577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.214578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.214697 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.214991 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.214993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.214994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.215232 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.215345 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.215822 (+ 477us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.215974 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:11:24.216130 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.201729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:37601 (local address 127.0.0.1:58354)
0924 16:11:24.202277 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.202332 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.203291 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.203777 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.203786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.204242 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.204798 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.204812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.206065 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.206072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.207396 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.207413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.208073 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.208077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.208184 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.208725 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.208743 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.210749 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.213782 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.213792 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.213799 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.214457 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.214794 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.214797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.214799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.214893 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.215345 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.215354 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.215488 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.215683 (+ 195us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.215880 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":1,"thread_start_us":140,"threads_started":1}
I20260924 16:11:24.217440 3454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:24.218248 3379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:24.218995 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.218749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47880)
0924 16:11:24.218874 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.218925 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 16:11:24.219755 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.219470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47892)
0924 16:11:24.219619 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.219669 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260924 16:11:24.219906 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
I20260924 16:11:24.222154 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:38257
--webserver_interface=127.13.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:11:24.226933 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.218016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.13.8.125:43763)
0924 16:11:24.218140 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.218146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.218161 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:11:24.218359 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.218361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.218394 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.218456 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.218940 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.219690 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.220654 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.220877 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.220984 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.223430 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.223433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.223435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.223455 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.225114 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.225612 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.225614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.225615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.225684 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.225980 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.225982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.225984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.226143 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.226276 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.226586 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.226735 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":43}
I20260924 16:11:24.226964 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.217934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46874)
0924 16:11:24.218059 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.218077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.218251 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.218503 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.218506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.218699 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.218837 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.218845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.219856 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.219862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.220514 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.220524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.220983 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.220986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.221103 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.221609 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.221622 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.223295 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.225245 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.225251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.225256 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.225516 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.225777 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.225783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.225788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.225889 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.226278 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.226284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.226399 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.226521 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.226682 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 16:11:24.227862 3454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:24.228554 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.228312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47902)
0924 16:11:24.228435 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.228507 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 16:11:24.229104 3405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.230010 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.229772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47910)
0924 16:11:24.229875 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.229917 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 16:11:24.230139 3454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.251063 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 1
I20260924 16:11:24.251933 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.251634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47922)
0924 16:11:24.251792 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.251852 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:11:24.252698 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.252499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47932)
0924 16:11:24.252592 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.252643 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 16:11:24.252851 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.279457 3454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 1
I20260924 16:11:24.280390 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.280084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47936)
0924 16:11:24.280246 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.280308 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 16:11:24.281073 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.280872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47938)
0924 16:11:24.280970 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.281021 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 16:11:24.281194 3454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.328001 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 2
I20260924 16:11:24.328907 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.328553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47948)
0924 16:11:24.328739 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.328815 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 16:11:24.329669 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.329415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47952)
0924 16:11:24.329541 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.329597 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 16:11:24.329830 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
W20260924 16:11:24.344537 3460 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:24.344827 3460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:24.344910 3460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:24.345008 3460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:24.349098 3460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:24.349162 3460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:24.349200 3460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:24.349238 3460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:24.349269 3460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:24.349365 3454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 2
I20260924 16:11:24.350266 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.349922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47958)
0924 16:11:24.350072 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.350133 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260924 16:11:24.350931 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.350735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47964)
0924 16:11:24.350834 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.350878 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 16:11:24.351068 3454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.355427 3460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:37601,127.13.8.125:43763,127.13.8.124:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:24.356598 3460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:24.357643 3460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:24.366351 3469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:24.366483 3467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:24.366582 3466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:24.366712 3460 server_base.cc:1034] running on GCE node
I20260924 16:11:24.367364 3460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:24.368558 3460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:24.369771 3460 hybrid_clock.cc:648] HybridClock initialized: now 1790266284369751 us; error 31 us; skew 500 ppm
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:24.372551 3460 init.cc:377] Logged in from keytab as kudu/127.13.8.124@KRBTEST.COM (short username kudu)
I20260924 16:11:24.373591 3460 webserver.cc:460] Webserver started at http://127.13.8.124:38579/ using document root <none> and password file <none>
I20260924 16:11:24.374297 3460 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:24.374385 3460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:24.374607 3460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:24.376704 3460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data/instance:
uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "ac9e8ec9cf5123c1ed994f1bfac2b916"
server_key_iv: "54495301bfd607ec7d1e29ad426545af"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.377276 3460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal/instance:
uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "ac9e8ec9cf5123c1ed994f1bfac2b916"
server_key_iv: "54495301bfd607ec7d1e29ad426545af"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.381244 3460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 16:11:24.383895 3476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:24.385202 3460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:11:24.385362 3460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd"
format_stamp: "Formatted at 2026-09-24 16:11:24 on dist-test-slave-vj6n"
server_key: "ac9e8ec9cf5123c1ed994f1bfac2b916"
server_key_iv: "54495301bfd607ec7d1e29ad426545af"
server_key_version: "encryptionkey@0"
I20260924 16:11:24.385491 3460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:24.424769 3460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:24.427948 3460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:24.428160 3460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:24.428999 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 3
I20260924 16:11:24.429919 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.429572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47972)
0924 16:11:24.429776 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.429842 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260924 16:11:24.430675 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.430452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:47976)
0924 16:11:24.430572 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.430622 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 16:11:24.430797 3379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:38257: Network error: Client connection negotiation failed: client connection to 127.13.8.124:38257: connect: Connection refused (error 111)
I20260924 16:11:24.435725 3460 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.124:38257
I20260924 16:11:24.435993 3528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.124:38257 every 8 connection(s)
I20260924 16:11:24.436789 3460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
I20260924 16:11:24.439777 3529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:24.439915 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3460
I20260924 16:11:24.440050 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790266001279336-13345-0/minicluster-data/master-2/wal/instance
I20260924 16:11:24.440338 13345 external_mini_cluster.cc:1428] Setting key 86b4a4e3e57b09ebc7b36531d0e8933c
I20260924 16:11:24.442262 3454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 3
I20260924 16:11:24.445426 3529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:11:24.454298 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.442750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:48000)
0924 16:11:24.442864 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.442878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.442992 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.443645 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.443649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.443864 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.443983 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.443992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.445606 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.445610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.446469 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.446481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.447283 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.447287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.447370 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.447891 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.447911 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.449228 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.452181 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.452185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.452187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.452456 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.452848 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.452851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.452853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.452966 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.453728 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.453731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.453840 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.453958 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.454100 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 16:11:24.455348 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.441672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.13.8.124:38257)
0924 16:11:24.442068 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.442084 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.442132 (+ 48us) server_negotiation.cc:407] Connection header received
0924 16:11:24.442999 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.443023 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.443367 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.443712 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.444493 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.445418 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.446645 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.447167 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.447662 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.450055 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.450059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.450061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.450097 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.452084 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.452988 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.452993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.452999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.453093 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.453439 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.453442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.453444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.453708 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.454705 (+ 997us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.454790 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.454931 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":251,"spinlock_wait_cycles":36096,"thread_start_us":118,"threads_started":1}
I20260924 16:11:24.455354 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.442793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.13.8.124:38257)
0924 16:11:24.443192 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.443200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.443213 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:11:24.443247 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.443250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.443367 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.444098 (+ 731us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.444319 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.445442 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.446607 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.447151 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.447329 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.449440 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.449449 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.449468 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.449506 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.452073 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.452573 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.452579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.452586 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.452760 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.453089 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.453091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.453093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.453596 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.453744 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.454134 (+ 390us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.454480 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":321,"thread_start_us":147,"threads_started":1}
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:24.459911 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.447326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:37601 (local address 127.0.0.1:58358)
0924 16:11:24.449147 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.449187 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.449324 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.449640 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.449649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.450132 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.450323 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.450335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.451335 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.451340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.452203 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.452214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.452681 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.452687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.452840 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.453445 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.453462 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.456189 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.458154 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.458163 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.458169 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.458505 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.458797 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.458802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.458804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.458887 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.459294 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.459303 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.459418 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.459549 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.459740 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"thread_start_us":87,"threads_started":1}
I20260924 16:11:24.459897 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.447932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.13.8.126:37601)
0924 16:11:24.448050 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.448053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.449268 (+ 1215us) server_negotiation.cc:407] Connection header received
0924 16:11:24.449430 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.449433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.449506 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.449593 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.450462 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.451222 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.452322 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.452572 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.452679 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.456371 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.456375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.456377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.456406 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.458037 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.458617 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.458620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.458622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.458706 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.458982 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.458985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.458987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.459176 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.459291 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.459631 (+ 340us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.459739 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:11:24.460408 3454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: Bootstrap starting.
I20260924 16:11:24.460927 3529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:24.463349 3454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: Neither blocks nor log segments found. Creating new log.
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:24.464027 3454 log.cc:824] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:24.466464 3454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: No bootstrap required, opened a new log
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:11:24.468315 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.458156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.13.8.126:37601)
0924 16:11:24.458496 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.458512 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.458528 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:11:24.458563 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.458565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.458638 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.459522 (+ 884us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.459658 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.460414 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.461316 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.461670 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.462152 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.464428 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.464432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.464435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.464462 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.465937 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.466416 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.466420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.466422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.466504 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.466812 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.466816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.466817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.467034 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.467886 (+ 852us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.467970 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.468093 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
I20260924 16:11:24.471614 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.461409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:43763 (local address 127.0.0.1:46878)
0924 16:11:24.461557 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.461570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.461642 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.461859 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.461861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.462037 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.462139 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.462148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.463123 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.463128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.464013 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.464029 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.464758 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.464761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.464841 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.466341 (+ 1500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.466355 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.467766 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.469986 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.469991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.469993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.470183 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.470506 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.470509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.470511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.470592 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.471127 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.471130 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.471231 (+ 101us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.471335 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.471465 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260924 16:11:24.471709 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.461485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.13.8.125:43763)
0924 16:11:24.461596 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.461602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.461616 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:11:24.461716 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.461721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.461760 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.461844 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.462315 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.462997 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.464152 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.464486 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.464643 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.468000 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.468004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.468007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.468030 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.469877 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.470321 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.470326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.470328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.470409 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.470686 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.470691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.470693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.471001 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.471120 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.471447 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.471561 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":43}
I20260924 16:11:24.471814 3454 raft_consensus.cc:348] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.472051 3454 raft_consensus.cc:729] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d583d3b64aec429e88e910adbcf58d83, State: Initialized, Role: FOLLOWER
I20260924 16:11:24.472483 3454 consensus_queue.cc:260] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.472983 3529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:24.472982 3538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.473380 3538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.474391 3454 sys_catalog.cc:564] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
W20260924 16:11:24.479583 3549 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:24.479658 3549 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:24.481400 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.469783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46880 (local address 127.13.8.125:43763)
0924 16:11:24.470458 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.470467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.470480 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:11:24.470515 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.470518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.470568 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.470701 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.471428 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.472132 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.473101 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.473479 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.476418 (+ 2939us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.476520 (+ 102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.476523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.476526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.476541 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.478666 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.479189 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.479192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.479194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.479299 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.479942 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.479947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.479949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.480190 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.480990 (+ 800us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.481070 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.481192 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":600,"thread_start_us":112,"threads_started":1}
I20260924 16:11:24.477892 3386 master_runner.cc:186] Error getting master registration for 127.13.8.126:37601: OK, instance_id { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" instance_seqno: 1790266283958703 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 16:11:24.482005 3386 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:11:24.483158 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.473478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:48004)
0924 16:11:24.473580 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.473595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.473671 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.474098 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.474103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.474320 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.474425 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.474433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.475490 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.475493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.476315 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.476329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.477033 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.477041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.477135 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.477580 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.477598 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.479093 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.481070 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.481073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.481077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.481318 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.481840 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.481842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.481845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.482025 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.482644 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.482646 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.482754 (+ 108us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.482867 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.482998 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 16:11:24.483166 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.473674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.13.8.124:38257)
0924 16:11:24.473767 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.473773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.473789 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:11:24.473827 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.473832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.473906 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.473977 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.474553 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.475380 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.476428 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.476809 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.476911 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.479227 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.479232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.479235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.479259 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.480965 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.481419 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.481422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.481424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.481738 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.482198 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.482201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.482235 (+ 34us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.482508 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.482595 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.482908 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.483053 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":34}
I20260924 16:11:24.484980 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 16:11:24.486150 3529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: Bootstrap starting.
I20260924 16:11:24.486382 3348 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8d6d26d64b0e4101bfff381d70b02d6f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58366
I20260924 16:11:24.488901 3529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:24.489673 3529 log.cc:824] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:24.491940 3529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: No bootstrap required, opened a new log
I20260924 16:11:24.496098 3529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.496380 3529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45f6bfd7ac594006bc956f2b6ffcbdbd, State: Initialized, Role: FOLLOWER
I20260924 16:11:24.496698 3529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.497166 3552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.497292 3552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.497974 3529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: configured and running, proceeding with master startup.
W20260924 16:11:24.503525 3563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:24.503605 3563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:24.501734 3460 master_runner.cc:186] Error getting master registration for 127.13.8.126:37601: OK, instance_id { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" instance_seqno: 1790266283958703 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 16:11:24.504686 3460 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:11:24.594913 3379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } attempt: 4
Sep 24 16:11:24 dist-test-slave-vj6n krb5kdc[3297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:11:24.605064 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.595633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.13.8.124:38257)
0924 16:11:24.595769 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.595775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.595844 (+ 69us) server_negotiation.cc:407] Connection header received
0924 16:11:24.596017 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.596021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.596072 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.596160 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.596728 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.597554 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.598510 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.598955 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.599057 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.601647 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.601651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.601653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.601684 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.603415 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.603826 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.603830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.603832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.603917 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.604175 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.604180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.604181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.604412 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.604511 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.604793 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.604906 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 16:11:24.605064 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.595548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:38257 (local address 127.0.0.1:48010)
0924 16:11:24.595769 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:24.595784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:24.595900 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:24.596182 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:24.596186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:24.596425 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:24.596559 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.596569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.597667 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.597670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.598396 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:24.598405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.599251 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.599257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:24.599352 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.599924 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:24.599941 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:24.601506 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:24.603512 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.603515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.603518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.603723 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.603997 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:24.604000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:24.604002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:24.604085 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:24.604512 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:24.604516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:24.604615 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 16:11:24.604722 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 16:11:24.604874 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260924 16:11:24.608958 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: Bootstrap starting.
I20260924 16:11:24.611931 3379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:24.612828 3379 log.cc:824] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:24.615192 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: No bootstrap required, opened a new log
I20260924 16:11:24.618484 3379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.618736 3379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6d26d64b0e4101bfff381d70b02d6f, State: Initialized, Role: FOLLOWER
I20260924 16:11:24.619091 3379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.619633 3566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.619903 3566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.620543 3379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:24.620605 3348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 16:11:24.620699 3348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:24.622301 3348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.623375 3348 leader_election.cc:288] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [CANDIDATE]: Term 1 election: Requested vote from peers d583d3b64aec429e88e910adbcf58d83 (127.13.8.125:43763), 45f6bfd7ac594006bc956f2b6ffcbdbd (127.13.8.124:38257)
I20260924 16:11:24.624640 3428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d583d3b64aec429e88e910adbcf58d83"
I20260924 16:11:24.624935 3428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:24.626425 3428 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d6d26d64b0e4101bfff381d70b02d6f in term 1.
I20260924 16:11:24.626520 3503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd"
I20260924 16:11:24.626816 3503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:24.627781 3503 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d6d26d64b0e4101bfff381d70b02d6f in term 1.
I20260924 16:11:24.628250 3329 leader_election.cc:302] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45f6bfd7ac594006bc956f2b6ffcbdbd, 8d6d26d64b0e4101bfff381d70b02d6f; no voters:
I20260924 16:11:24.628786 3566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:24.629076 3566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [term 1 LEADER]: Becoming Leader. State: Replica: 8d6d26d64b0e4101bfff381d70b02d6f, State: Running, Role: LEADER
I20260924 16:11:24.629457 3566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } }
I20260924 16:11:24.632493 3582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6d26d64b0e4101bfff381d70b02d6f. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.632601 3582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: This master's current role is: LEADER
W20260924 16:11:24.635308 3577 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:24.635380 3577 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:24.638417 3587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:24.639387 3587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 16:11:24.650800 3503 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [term 1 FOLLOWER]: Refusing update from remote peer 8d6d26d64b0e4101bfff381d70b02d6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:11:24.651829 3566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:11:24.656529 3552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6d26d64b0e4101bfff381d70b02d6f. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.656662 3552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.656911 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.631169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.13.8.125:43763)
0924 16:11:24.631303 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.631307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.634483 (+ 3176us) server_negotiation.cc:407] Connection header received
0924 16:11:24.635262 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.635265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.635327 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.635384 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.636469 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.637221 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.640599 (+ 3378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.640953 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.641040 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.652303 (+ 11263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.652307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.652311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.652345 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.654208 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.654760 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.654764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.654766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.654854 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.655443 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.655449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.655451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.655673 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.655814 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.656594 (+ 780us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.656746 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:11:24.658303 3428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [term 1 FOLLOWER]: Refusing update from remote peer 8d6d26d64b0e4101bfff381d70b02d6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:11:24.658993 3566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:11:24.661191 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.631643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.13.8.124:38257)
0924 16:11:24.631753 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.631758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.635064 (+ 3306us) server_negotiation.cc:407] Connection header received
0924 16:11:24.650353 (+ 15289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.650357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.650422 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.650486 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.651462 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.652243 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.653423 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.653760 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.653841 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.655593 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.655598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.655601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.655629 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.658157 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.659666 (+ 1509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.659670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.659672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.659752 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.660008 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.660011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.660013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.660200 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.660332 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.660934 (+ 602us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.661041 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:24.662999 3566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.663132 3566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:24.663735 3589 mvcc.cc:205] Tried to move back new op lower bound from 7332930701918867456 to 7332930701846142976. Current Snapshot: MvccSnapshot[applied={T|T < 7332930701918867456}]
I20260924 16:11:24.664018 3538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6d26d64b0e4101bfff381d70b02d6f. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.664145 3538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.664217 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:24.630696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.13.8.126:37601)
0924 16:11:24.630813 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.630817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.634700 (+ 3883us) server_negotiation.cc:407] Connection header received
0924 16:11:24.636524 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.636528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.636593 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.636647 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.638725 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.639428 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.642005 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:24.642371 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:24.642450 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:24.656148 (+ 13698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:24.656152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:24.656155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:24.656182 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:24.659123 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.662546 (+ 3423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.662550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.662552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.662652 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:24.662913 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:24.662916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:24.662918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:24.663119 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:24.663244 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:24.663925 (+ 681us) server_negotiation.cc:299] Negotiation successful
0924 16:11:24.664068 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:24.664453 3552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.664552 3552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.665697 3587 catalog_manager.cc:1347] Generated new cluster ID: 41d09360e66f4297b35b09ce766803de
I20260924 16:11:24.665756 3587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:24.674333 3590 mvcc.cc:205] Tried to move back new op lower bound from 7332930701918867456 to 7332930701846142976. Current Snapshot: MvccSnapshot[applied={T|T < 7332930701918867456}]
I20260924 16:11:24.677230 3582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.677361 3582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:24.678593 3538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d6d26d64b0e4101bfff381d70b02d6f" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 37601 } } peers { permanent_uuid: "d583d3b64aec429e88e910adbcf58d83" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 43763 } } peers { permanent_uuid: "45f6bfd7ac594006bc956f2b6ffcbdbd" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 38257 } } }
I20260924 16:11:24.678710 3538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:24.680321 3592 mvcc.cc:205] Tried to move back new op lower bound from 7332930701918867456 to 7332930701846142976. Current Snapshot: MvccSnapshot[applied={T|T < 7332930701918867456}]
I20260924 16:11:24.721949 3587 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:24.723412 3587 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:24.743103 3587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d6d26d64b0e4101bfff381d70b02d6f: Generated new TSK 0
I20260924 16:11:24.744011 3587 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:11:24.768000 3535 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.766586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.13.8.126:37601)
0924 16:11:24.766707 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.766713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.766838 (+ 125us) server_negotiation.cc:407] Connection header received
0924 16:11:24.767030 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.767034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.767114 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.767184 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.767860 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58388: BlockingRecv error: recv got EOF from 127.0.0.1:58388 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260924 16:11:24.768206 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.766193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.13.8.125:43763)
0924 16:11:24.766383 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.766388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.767111 (+ 723us) server_negotiation.cc:407] Connection header received
0924 16:11:24.767309 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.767312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.767381 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.767434 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.768072 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46904: BlockingRecv error: recv got EOF from 127.0.0.1:46904 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260924 16:11:24.768208 3533 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.766742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.13.8.124:38257)
0924 16:11:24.766850 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.766855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.767014 (+ 159us) server_negotiation.cc:407] Connection header received
0924 16:11:24.767286 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.767289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.767372 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.767432 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.768073 (+ 641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48030: BlockingRecv error: recv got EOF from 127.0.0.1:48030 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260924 16:11:24.771628 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.770592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46912 (local address 127.13.8.125:43763)
0924 16:11:24.770710 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.770714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.770818 (+ 104us) server_negotiation.cc:407] Connection header received
0924 16:11:24.770952 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.770954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.771020 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.771087 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.771514 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46912: BlockingRecv error: recv got EOF from 127.0.0.1:46912 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 16:11:24.775539 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.774750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.13.8.125:43763)
0924 16:11:24.774854 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.774858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.774870 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:11:24.774916 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.774919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.774965 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.775067 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.775419 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46924: BlockingRecv error: recv got EOF from 127.0.0.1:46924 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260924 16:11:24.780398 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.779416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.13.8.125:43763)
0924 16:11:24.779528 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.779532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.779636 (+ 104us) server_negotiation.cc:407] Connection header received
0924 16:11:24.779771 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.779773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.779812 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.779875 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.780296 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46940: BlockingRecv error: recv got EOF from 127.0.0.1:46940 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260924 16:11:24.787603 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.786436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.13.8.125:43763)
0924 16:11:24.786572 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.786575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.786802 (+ 227us) server_negotiation.cc:407] Connection header received
0924 16:11:24.786948 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.786951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.787002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.787084 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.787510 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46942: BlockingRecv error: recv got EOF from 127.0.0.1:46942 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260924 16:11:24.798552 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.797453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.13.8.125:43763)
0924 16:11:24.797585 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.797589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.797714 (+ 125us) server_negotiation.cc:407] Connection header received
0924 16:11:24.797830 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.797833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.797890 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.797994 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.798440 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46958: BlockingRecv error: recv got EOF from 127.0.0.1:46958 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260924 16:11:24.818164 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.817086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46970 (local address 127.13.8.125:43763)
0924 16:11:24.817235 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.817240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.817296 (+ 56us) server_negotiation.cc:407] Connection header received
0924 16:11:24.817436 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.817440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.817499 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.817595 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.818041 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46970: BlockingRecv error: recv got EOF from 127.0.0.1:46970 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260924 16:11:24.853508 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.852374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.13.8.125:43763)
0924 16:11:24.852525 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.852528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.852667 (+ 139us) server_negotiation.cc:407] Connection header received
0924 16:11:24.852838 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.852841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.852923 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.853005 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.853410 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46978: BlockingRecv error: recv got EOF from 127.0.0.1:46978 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260924 16:11:24.921384 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:24.919941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.13.8.125:43763)
0924 16:11:24.920163 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:24.920169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:24.920287 (+ 118us) server_negotiation.cc:407] Connection header received
0924 16:11:24.920499 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:24.920509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:24.920573 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:24.920683 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:24.921234 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46988: BlockingRecv error: recv got EOF from 127.0.0.1:46988 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260924 16:11:25.052836 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:25.051687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.13.8.125:43763)
0924 16:11:25.051861 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.051865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.051954 (+ 89us) server_negotiation.cc:407] Connection header received
0924 16:11:25.052038 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.052041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.052091 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.052166 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:25.052708 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47004: BlockingRecv error: recv got EOF from 127.0.0.1:47004 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260924 16:11:25.312189 3537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:25.311174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.13.8.125:43763)
0924 16:11:25.311307 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.311311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.311396 (+ 85us) server_negotiation.cc:407] Connection header received
0924 16:11:25.311537 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.311539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.311588 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.311669 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:25.312090 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47016: BlockingRecv error: recv got EOF from 127.0.0.1:47016 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 16:11:25.481089 3549 catalog_manager.cc:1259] Loaded cluster ID: 41d09360e66f4297b35b09ce766803de
I20260924 16:11:25.481174 3549 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: loading cluster ID for follower catalog manager: success
I20260924 16:11:25.482931 3549 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: acquiring CA information for follower catalog manager: success
I20260924 16:11:25.484112 3549 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d583d3b64aec429e88e910adbcf58d83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:11:25.504985 3563 catalog_manager.cc:1259] Loaded cluster ID: 41d09360e66f4297b35b09ce766803de
I20260924 16:11:25.505095 3563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: loading cluster ID for follower catalog manager: success
I20260924 16:11:25.507027 3563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: acquiring CA information for follower catalog manager: success
I20260924 16:11:25.508226 3563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45f6bfd7ac594006bc956f2b6ffcbdbd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:11:25.830509 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:25.826326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.13.8.125:43763)
0924 16:11:25.826576 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.826581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.826698 (+ 117us) server_negotiation.cc:407] Connection header received
0924 16:11:25.826886 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.826889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.827040 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.827167 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:25.827422 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.827961 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.828853 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.829108 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.829243 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:25.829591 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:25.830062 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:25.830240 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:25.830277 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 16:11:25.830338 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260924 16:11:25.837201 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:25.833804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.13.8.124:38257)
0924 16:11:25.834037 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.834040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.834053 (+ 13us) server_negotiation.cc:407] Connection header received
0924 16:11:25.834136 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.834138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.834282 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.834378 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:25.834575 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.835112 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.835877 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.836101 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.836176 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:25.836445 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:25.836887 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:25.836980 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:25.837027 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 16:11:25.837074 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260924 16:11:25.844728 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:25.840112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.13.8.126:37601)
0924 16:11:25.840380 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.840385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.840641 (+ 256us) server_negotiation.cc:407] Connection header received
0924 16:11:25.840807 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.840812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.841022 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.841168 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:25.841556 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.842359 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.843192 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.843486 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.843588 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:25.843878 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:25.844241 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:25.844334 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:25.844491 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 16:11:25.844554 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260924 16:11:25.844978 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:25.840114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.13.8.125:43763)
0924 16:11:25.840246 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.840251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.840889 (+ 638us) server_negotiation.cc:407] Connection header received
0924 16:11:25.841052 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.841057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.841116 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.841187 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:25.841460 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.842017 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.843472 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.843832 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.843937 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:25.844169 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:25.844255 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:25.844339 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:25.844763 (+ 424us) server_negotiation.cc:299] Negotiation successful
0924 16:11:25.844822 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:25.845389 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:25.840334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.13.8.124:38257)
0924 16:11:25.840444 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:25.840448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:25.840789 (+ 341us) server_negotiation.cc:407] Connection header received
0924 16:11:25.840967 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:25.840970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:25.841018 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:25.841088 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:25.841315 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.842098 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.844009 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:25.844314 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:25.844405 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:25.844779 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:25.844856 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:25.844924 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:25.845194 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 16:11:25.845255 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:25.847940 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3310
I20260924 16:11:25.855417 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3386
I20260924 16:11:25.861439 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3460
2026-09-24T16:11:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3266 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 16:11:25 dist-test-slave-vj6n krb5kdc[3679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 16:11:25 dist-test-slave-vj6n krb5kdc[3679](info): set up 2 sockets
Sep 24 16:11:25 dist-test-slave-vj6n krb5kdc[3679](info): commencing operation
krb5kdc: starting...
W20260924 16:11:26.907618 13345 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/test-user.keytab.
Sep 24 16:11:26 dist-test-slave-vj6n krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T16:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T16:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
Entry for principal kudu/127.13.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab.
I20260924 16:11:27.031365 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:33925
--webserver_interface=127.13.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 16:11:27.143253 3692 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:27.143534 3692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:27.143558 3692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:27.143618 3692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:27.147557 3692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:27.147614 3692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:27.147634 3692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:27.147650 3692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:27.147665 3692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:27.152783 3692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.126.keytab
--principal=kudu/127.13.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.126:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:27.153795 3692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:27.154847 3692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:27.163290 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:27.163332 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:27.163312 3692 server_base.cc:1034] running on GCE node
W20260924 16:11:27.163301 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:27.164191 3692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:27.165282 3692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:27.166460 3692 hybrid_clock.cc:648] HybridClock initialized: now 1790266287166445 us; error 25 us; skew 500 ppm
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:27.169094 3692 init.cc:377] Logged in from keytab as kudu/127.13.8.126@KRBTEST.COM (short username kudu)
I20260924 16:11:27.169917 3692 webserver.cc:460] Webserver started at http://127.13.8.126:41287/ using document root <none> and password file <none>
I20260924 16:11:27.170548 3692 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:27.170610 3692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:27.170774 3692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:27.172770 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data/instance:
uuid: "0097e28e66fd4ac5899095faa87f950b"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "caa02c905f3dd58c8b508ea77061eab5"
server_key_iv: "b876cda6d64897af33e90693f10c22c1"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.173305 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance:
uuid: "0097e28e66fd4ac5899095faa87f950b"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "caa02c905f3dd58c8b508ea77061eab5"
server_key_iv: "b876cda6d64897af33e90693f10c22c1"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.176936 3692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260924 16:11:27.179369 3707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:27.180627 3692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 16:11:27.180732 3692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal
uuid: "0097e28e66fd4ac5899095faa87f950b"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "caa02c905f3dd58c8b508ea77061eab5"
server_key_iv: "b876cda6d64897af33e90693f10c22c1"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.180815 3692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:27.192584 3692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:27.195447 3692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:27.195598 3692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:27.202657 3692 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.126:33925
I20260924 16:11:27.202973 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.126:33925 every 8 connection(s)
I20260924 16:11:27.203683 3692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/data/info.pb
I20260924 16:11:27.206722 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:27.207527 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3692
I20260924 16:11:27.207671 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-0/wal/instance
I20260924 16:11:27.207952 13345 external_mini_cluster.cc:1428] Setting key e08a06ba7517ffa6a17aa48d5a4bc09f
I20260924 16:11:27.211340 3760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.125@KRBTEST.COM" created.
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:27.227939 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.214369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33925 (local address 127.0.0.1:33736)
0924 16:11:27.215073 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.215124 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.216043 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.216851 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.216857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.217303 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.217623 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.217638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.218900 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.218908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.219683 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.219693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.220327 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.220331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.220436 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.221063 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.221087 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.222916 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.224934 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.224944 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.224955 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.225296 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.225622 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.225625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.225627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.225778 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.226444 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.226452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.226624 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.226819 (+ 195us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.227195 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":53,"thread_start_us":170,"threads_started":1}
I20260924 16:11:27.227952 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.214369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.13.8.126:33925)
0924 16:11:27.214847 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.214863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.215258 (+ 395us) server_negotiation.cc:407] Connection header received
0924 16:11:27.216450 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.216455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.216669 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.216855 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.217769 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.218761 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.219800 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.220167 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.220340 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.223107 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.223113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.223128 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.223150 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.224817 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.225410 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.225415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.225417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.225523 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.225881 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.225884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.225885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.226319 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.226426 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.226873 (+ 447us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.227110 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":304,"thread_start_us":144,"threads_started":1}
I20260924 16:11:27.230502 3760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:27.231236 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.230975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:55610)
0924 16:11:27.231086 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.231167 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:11:27.231452 3708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
I20260924 16:11:27.232286 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.232104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:55616)
0924 16:11:27.232195 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.232238 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 16:11:27.232415 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:34157: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
Entry for principal kudu/127.13.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab.
I20260924 16:11:27.236689 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:34157
--webserver_interface=127.13.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 16:11:27.281711 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } attempt: 1
I20260924 16:11:27.282697 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.282345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:55618)
0924 16:11:27.282511 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.282589 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:11:27.283483 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.283252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:55630)
0924 16:11:27.283353 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.283423 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 16:11:27.283620 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:34157: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
I20260924 16:11:27.351800 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } attempt: 2
I20260924 16:11:27.352747 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.352383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:51194)
0924 16:11:27.352558 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.352650 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260924 16:11:27.353482 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.353235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:51208)
0924 16:11:27.353332 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.353403 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 16:11:27.353626 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.125:34157: Network error: Client connection negotiation failed: client connection to 127.13.8.125:34157: connect: Connection refused (error 111)
W20260924 16:11:27.354552 3767 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:27.354857 3767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:27.354905 3767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:27.355015 3767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:27.359112 3767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:27.359172 3767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:27.359218 3767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:27.359279 3767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:27.359301 3767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:27.364722 3767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.125.keytab
--principal=kudu/127.13.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.125:34157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:27.365839 3767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:27.366926 3767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:27.375881 3772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:27.375936 3775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:27.375890 3773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:27.376077 3767 server_base.cc:1034] running on GCE node
I20260924 16:11:27.376835 3767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:27.377955 3767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:27.379117 3767 hybrid_clock.cc:648] HybridClock initialized: now 1790266287379079 us; error 49 us; skew 500 ppm
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:27.382011 3767 init.cc:377] Logged in from keytab as kudu/127.13.8.125@KRBTEST.COM (short username kudu)
I20260924 16:11:27.383065 3767 webserver.cc:460] Webserver started at http://127.13.8.125:45443/ using document root <none> and password file <none>
I20260924 16:11:27.383726 3767 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:27.383826 3767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:27.384045 3767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:27.386317 3767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data/instance:
uuid: "456825c477ba4fbebe5a079071af8322"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "dd8706826f49c23c218817d4d3761a49"
server_key_iv: "ae350c4d1f3274fe55a495704e72efcc"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.386898 3767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal/instance:
uuid: "456825c477ba4fbebe5a079071af8322"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "dd8706826f49c23c218817d4d3761a49"
server_key_iv: "ae350c4d1f3274fe55a495704e72efcc"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.390828 3767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 16:11:27.393383 3782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:27.394763 3767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 16:11:27.394912 3767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal
uuid: "456825c477ba4fbebe5a079071af8322"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "dd8706826f49c23c218817d4d3761a49"
server_key_iv: "ae350c4d1f3274fe55a495704e72efcc"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.395035 3767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:27.421108 3767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:27.424661 3767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:27.424868 3767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:27.432197 3767 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.125:34157
I20260924 16:11:27.432482 3834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.125:34157 every 8 connection(s)
I20260924 16:11:27.433300 3767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/data/info.pb
I20260924 16:11:27.433532 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3767
I20260924 16:11:27.433655 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-1/wal/instance
I20260924 16:11:27.434000 13345 external_mini_cluster.cc:1428] Setting key f7ad2ca84563e8160ba23dfef95c3063
I20260924 16:11:27.436452 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:27.440454 3835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.8.124@KRBTEST.COM" created.
I20260924 16:11:27.444820 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } attempt: 3
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:27.456771 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.445310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:51218)
0924 16:11:27.445455 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.445473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.445567 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.446254 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.446258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.446466 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.446600 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.446608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.448167 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.448171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.448704 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.448713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.449492 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.449499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.449687 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.450384 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.450402 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.452022 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.454519 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.454525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.454528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.454811 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.455223 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.455226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.455228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.455333 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.456089 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.456092 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.456195 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.456321 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.456498 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":27}
I20260924 16:11:27.457370 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.442528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.13.8.126:33925)
0924 16:11:27.442668 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.442673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.443657 (+ 984us) server_negotiation.cc:407] Connection header received
0924 16:11:27.444660 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.444664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.444729 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.444829 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.446926 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.447729 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.449368 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.449794 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.449912 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.452822 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.452830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.452833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.452877 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.454524 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.455044 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.455047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.455049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.455126 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.455486 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.455492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.455496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.455743 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.455843 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.457108 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.457237 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":46}
I20260924 16:11:27.457859 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.445488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.13.8.125:34157)
0924 16:11:27.445821 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.445833 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.445855 (+ 22us) server_negotiation.cc:407] Connection header received
0924 16:11:27.445893 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.445897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.446068 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.446275 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.446750 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.448040 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.448830 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.449356 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.449529 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.452209 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.452219 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.452237 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.452272 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.454404 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.454933 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.454938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.454939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.455096 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.455432 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.455435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.455451 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.455970 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.456104 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.456411 (+ 307us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.456703 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":224,"thread_start_us":108,"threads_started":1}
I20260924 16:11:27.457863 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.442914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33925 (local address 127.0.0.1:50236)
0924 16:11:27.443426 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.443502 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.444433 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.445011 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.445023 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.445941 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.446734 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.446754 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.447856 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.447862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.449212 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.449229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.449967 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.449972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.450083 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.450681 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.450699 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.452678 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.454622 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.454634 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.454646 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.454955 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.455223 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.455226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.455228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.455388 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.455877 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.455886 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.456097 (+ 211us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.456328 (+ 231us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.456791 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":183,"thread_start_us":126,"threads_started":1}
I20260924 16:11:27.459463 3835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:27.460125 3760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
Entry for principal kudu/127.13.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab.
I20260924 16:11:27.460917 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.460587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52718)
0924 16:11:27.460748 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.460842 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 16:11:27.461699 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.461391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52726)
0924 16:11:27.461564 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.461611 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260924 16:11:27.461858 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.461972 13345 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
/tmp/dist-test-taskf9hPi5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:39889
--webserver_interface=127.13.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.8.84:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:11:27.468568 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.460081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:51222)
0924 16:11:27.460503 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.460516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.460597 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.460858 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.460860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.461032 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.461127 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.461134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.461956 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.461963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.462694 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.462704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.463198 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.463202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.463297 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.463778 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.463790 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.465098 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.466979 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.466984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.466987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.467191 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.467486 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.467491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.467493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.467584 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.467971 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.467974 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.468070 (+ 96us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.468185 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.468334 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352}
I20260924 16:11:27.468568 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.460355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.13.8.125:34157)
0924 16:11:27.460486 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.460493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.460561 (+ 68us) server_negotiation.cc:407] Connection header received
0924 16:11:27.460710 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.460713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.460755 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.460849 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.461243 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.461819 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.462818 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.463070 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.463198 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.465228 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.465232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.465234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.465264 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.466847 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.467306 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.467310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.467312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.467389 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.467683 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.467686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.467688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.467869 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.467982 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.468232 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.468394 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":38}
I20260924 16:11:27.469689 3835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:27.470472 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.470193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52736)
0924 16:11:27.470343 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.470414 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260924 16:11:27.471014 3784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.471922 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.471720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52752)
0924 16:11:27.471823 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.471871 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 16:11:27.472075 3835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.493039 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 1
I20260924 16:11:27.494014 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.493676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52766)
0924 16:11:27.493866 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.493932 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 16:11:27.494779 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.494564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52776)
0924 16:11:27.494672 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.494724 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:11:27.494915 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.521333 3835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 1
I20260924 16:11:27.522338 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.521974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52784)
0924 16:11:27.522137 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.522205 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 16:11:27.523118 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.522890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52786)
0924 16:11:27.523005 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.523059 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 16:11:27.523276 3835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.570071 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 2
I20260924 16:11:27.570945 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.570654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52794)
0924 16:11:27.570797 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.570867 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:11:27.571621 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.571425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52806)
0924 16:11:27.571519 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.571568 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 16:11:27.571770 3760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
W20260924 16:11:27.580185 3841 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 16:11:27.580538 3841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 16:11:27.580597 3841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 16:11:27.580679 3841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 16:11:27.584930 3841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 16:11:27.585017 3841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 16:11:27.585055 3841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 16:11:27.585107 3841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 16:11:27.585144 3841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 16:11:27.590987 3841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.8.84:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.8.126:33925,127.13.8.125:34157,127.13.8.124:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/krb5kdc/kudu_127.13.8.124.keytab
--principal=kudu/127.13.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.8.124:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 24 Sep 2026 15:14:23 UTC on ccycloud.quasar-qkwurp.root.comops.site
build id 4670390
I20260924 16:11:27.591418 3835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 2
I20260924 16:11:27.592270 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.591975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52810)
0924 16:11:27.592124 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.592186 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 16:11:27.592391 3841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 16:11:27.592967 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.592761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52814)
0924 16:11:27.592865 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.592914 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 16:11:27.593137 3835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.8.124:39889: Network error: Client connection negotiation failed: client connection to 127.13.8.124:39889: connect: Connection refused (error 111)
I20260924 16:11:27.593603 3841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 16:11:27.602649 3850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 16:11:27.602653 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:27.602913 3841 server_base.cc:1034] running on GCE node
W20260924 16:11:27.602653 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 16:11:27.603823 3841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 16:11:27.605018 3841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 16:11:27.606266 3841 hybrid_clock.cc:648] HybridClock initialized: now 1790266287606203 us; error 33 us; skew 500 ppm
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 16:11:27.609427 3841 init.cc:377] Logged in from keytab as kudu/127.13.8.124@KRBTEST.COM (short username kudu)
I20260924 16:11:27.610479 3841 webserver.cc:460] Webserver started at http://127.13.8.124:37519/ using document root <none> and password file <none>
I20260924 16:11:27.611073 3841 fs_manager.cc:359] Metadata directory not provided
I20260924 16:11:27.611174 3841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 16:11:27.611414 3841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 16:11:27.613546 3841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data/instance:
uuid: "effca16e25954b7b98ee12fc839a4534"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "ea6a90a4f6d793f5af2cbad87ecdc1bb"
server_key_iv: "bd2b04591e0367d9463e88d1be9b5d7b"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.614138 3841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal/instance:
uuid: "effca16e25954b7b98ee12fc839a4534"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "ea6a90a4f6d793f5af2cbad87ecdc1bb"
server_key_iv: "bd2b04591e0367d9463e88d1be9b5d7b"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.618059 3841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260924 16:11:27.620852 3858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 16:11:27.622092 3841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260924 16:11:27.622287 3841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data,/tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal
uuid: "effca16e25954b7b98ee12fc839a4534"
format_stamp: "Formatted at 2026-09-24 16:11:27 on dist-test-slave-vj6n"
server_key: "ea6a90a4f6d793f5af2cbad87ecdc1bb"
server_key_iv: "bd2b04591e0367d9463e88d1be9b5d7b"
server_key_version: "encryptionkey@0"
I20260924 16:11:27.622404 3841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 16:11:27.635442 3841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 16:11:27.638939 3841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 16:11:27.639142 3841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 16:11:27.646194 3841 rpc_server.cc:307] RPC server started. Bound to: 127.13.8.124:39889
I20260924 16:11:27.646541 3911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.8.124:39889 every 8 connection(s)
I20260924 16:11:27.647248 3841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/data/info.pb
I20260924 16:11:27.649020 13345 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu as pid 3841
I20260924 16:11:27.649160 13345 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf9hPi5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790266001279336-13345-0/minicluster-data/master-2/wal/instance
I20260924 16:11:27.649473 13345 external_mini_cluster.cc:1428] Setting key c040ba8edcfdb9df850690f254e7eb91
I20260924 16:11:27.650374 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 16:11:27.654825 3912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
I20260924 16:11:27.664352 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.650857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.13.8.124:39889)
0924 16:11:27.651282 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.651298 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.651333 (+ 35us) server_negotiation.cc:407] Connection header received
0924 16:11:27.652003 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.652020 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.652373 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.652668 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.653590 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.654685 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.655832 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.656323 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.656518 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.659457 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.659462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.659471 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.659498 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.661351 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.661793 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.661798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.661803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.661917 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.662245 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.662249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.662250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.662663 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.662867 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.663200 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.663515 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260924 16:11:27.665956 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.656834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.13.8.126:33925)
0924 16:11:27.656986 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.656990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.657453 (+ 463us) server_negotiation.cc:407] Connection header received
0924 16:11:27.657603 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.657606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.657664 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.657734 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.658659 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.659206 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.660088 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.660324 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.660449 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.662646 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.662649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.662651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.662678 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.664115 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.664624 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.664628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.664630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.664727 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.665023 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.665026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.665028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.665228 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.665337 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.665666 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.665793 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 16:11:27.665978 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.656775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.126:33925 (local address 127.0.0.1:50244)
0924 16:11:27.657295 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.657354 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.657490 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.657838 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.657845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.658353 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.658534 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.658540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.659324 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.659328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.659958 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.659967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.660445 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.660448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.660563 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.661020 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.661032 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.662491 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.664244 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.664256 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.664262 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.664509 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.664837 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.664841 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.664843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.664929 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.665344 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.665352 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.665467 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.665608 (+ 141us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.665793 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":177,"threads_started":1}
I20260924 16:11:27.667026 3912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:27.670912 3760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 3
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.126@KRBTEST.COM
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.126@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:11:27.676267 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.666799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.13.8.126:33925)
0924 16:11:27.666914 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.666917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.666934 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:11:27.667061 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.667064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.667099 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.667194 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.668008 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.668493 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.669260 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.669471 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.669573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.672551 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.672554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.672557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.672580 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.674543 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.675015 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.675019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.675021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.675103 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.675386 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.675389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.675391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.675567 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.675881 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.675956 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.676076 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:27.676355 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.667598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.13.8.125:34157)
0924 16:11:27.667742 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.667748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.667762 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:11:27.667806 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.667808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.667844 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.667925 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.668309 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.668795 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.669533 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.669746 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.669850 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.672505 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.672512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.672518 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.672546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.674357 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.674859 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.674863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.674865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.674950 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.675321 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.675325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.675327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.675567 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.675642 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.676048 (+ 406us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.676189 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 16:11:27.677052 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.667479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.125:34157 (local address 127.0.0.1:51230)
0924 16:11:27.667589 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.667602 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.667680 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.667941 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.667944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.668104 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.668193 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.668198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.668901 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.668905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.669412 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.669419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.669854 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.669858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.669954 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.670354 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.670373 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.672355 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.674492 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.674499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.674503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.674751 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.675079 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.675085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.675090 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.675203 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.675724 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.675729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.675830 (+ 101us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.676491 (+ 661us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.676858 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 16:11:27.677804 3912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } has no permanent_uuid. Determining permanent_uuid...
I20260924 16:11:27.680066 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.671316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52830)
0924 16:11:27.671414 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.671429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.671500 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.671729 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.671731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.671861 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.671934 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.671938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.672791 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.672794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.673476 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.673489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.674079 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.674083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.674175 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.674626 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.674645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.676031 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.678377 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.678380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.678382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.678645 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.678959 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.678962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.678964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.679086 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.679462 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.679466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.679561 (+ 95us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.679664 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.679805 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 16:11:27.680094 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.671379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.13.8.124:39889)
0924 16:11:27.671506 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.671513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.671528 (+ 15us) server_negotiation.cc:407] Connection header received
0924 16:11:27.671567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.671571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.671625 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.671719 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.672042 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.672687 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.673620 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.673951 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.674066 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.676307 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.676311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.676313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.676337 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.678264 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.678747 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.678751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.678752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.678840 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.679205 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.679208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.679209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.679363 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.679467 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.679727 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.679832 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.124@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:11:27.684152 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: Bootstrap starting.
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.8.125@KRBTEST.COM
I20260924 16:11:27.684597 3835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } attempt: 3
I20260924 16:11:27.688279 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.678308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52846)
0924 16:11:27.678505 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.678522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.678573 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.679131 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.679145 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.679305 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.679385 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.679392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.680874 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.680877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.681678 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.681688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.682154 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.682158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.682252 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.682689 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.682702 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.684004 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.686496 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.686500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.686502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.686832 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.687175 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.687178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.687180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.687271 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.687766 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.687769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.687859 (+ 90us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.687968 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.688093 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260924 16:11:27.688387 3760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:27.688792 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.678452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.13.8.124:39889)
0924 16:11:27.678861 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.678872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.678886 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:11:27.678921 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.678925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.678969 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.679069 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.679496 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.680753 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.681805 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.682043 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.682153 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.684173 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.684177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.684180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.684204 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.686150 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.686951 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.686956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.686958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.687061 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.687379 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.687383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.687385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.687655 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.687766 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.688052 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.688199 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":127,"threads_started":1}
I20260924 16:11:27.689371 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.677999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.13.8.125:34157)
0924 16:11:27.678121 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.678125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.678139 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:11:27.678190 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.678195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.678288 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.678373 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.679503 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.680758 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.681806 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.682116 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.682241 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.685373 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.685378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.685382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.685415 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.687094 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.687646 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.687649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.687651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.687745 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.688124 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.688128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.688129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.688370 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.688609 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.689062 (+ 453us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.689194 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 16:11:27.690580 3760 log.cc:824] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:27.692528 3912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: Bootstrap starting.
I20260924 16:11:27.693552 13345 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Sep 24 16:11:27 dist-test-slave-vj6n krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790266287, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.8.125@KRBTEST.COM for kudu/127.13.8.124@KRBTEST.COM
I20260924 16:11:27.695541 3912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:27.694976 3734 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0097e28e66fd4ac5899095faa87f950b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50254
I20260924 16:11:27.696596 3912 log.cc:824] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:27.696931 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: No bootstrap required, opened a new log
I20260924 16:11:27.698292 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.685007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.8.124:39889 (local address 127.0.0.1:52858)
0924 16:11:27.685407 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0924 16:11:27.685422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 16:11:27.685497 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 16:11:27.689229 (+ 3732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 16:11:27.689233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 16:11:27.689443 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0924 16:11:27.689553 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.689561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.690843 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.690847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.691540 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 16:11:27.691551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.692120 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.692124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 16:11:27.692217 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.692659 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 16:11:27.692675 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 16:11:27.694267 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 16:11:27.696431 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.696437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.696439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.696725 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.697126 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 16:11:27.697130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 16:11:27.697132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 16:11:27.697231 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 16:11:27.697694 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 16:11:27.697697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 16:11:27.697814 (+ 117us) client_negotiation.cc:769] Sending connection context
0924 16:11:27.697928 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 16:11:27.698093 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323}
I20260924 16:11:27.698609 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.685093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.13.8.124:39889)
0924 16:11:27.688996 (+ 3903us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.689001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.689015 (+ 14us) server_negotiation.cc:407] Connection header received
0924 16:11:27.689050 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.689054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.689102 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.689191 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.689880 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.690645 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.691699 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.691992 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.692094 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.694448 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.694452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.694455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.694482 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.696268 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.696870 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.696874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.696876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.696970 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.697342 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.697345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.697347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.697591 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.697705 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.698096 (+ 391us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.698331 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3826}
I20260924 16:11:27.699648 3912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: No bootstrap required, opened a new log
I20260924 16:11:27.700331 3760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.700572 3760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0097e28e66fd4ac5899095faa87f950b, State: Initialized, Role: FOLLOWER
I20260924 16:11:27.700929 3760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.701840 3835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: Bootstrap starting.
I20260924 16:11:27.702359 3760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:27.702431 3734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 16:11:27.702548 3734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:27.703789 3734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.703805 3912 raft_consensus.cc:348] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.704010 3912 raft_consensus.cc:729] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: effca16e25954b7b98ee12fc839a4534, State: Initialized, Role: FOLLOWER
I20260924 16:11:27.704304 3912 consensus_queue.cc:260] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.705636 3912 sys_catalog.cc:564] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:27.705715 3886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0097e28e66fd4ac5899095faa87f950b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "effca16e25954b7b98ee12fc839a4534"
I20260924 16:11:27.705940 3886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:27.707952 3886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0097e28e66fd4ac5899095faa87f950b in term 1.
I20260924 16:11:27.708438 3709 leader_election.cc:302] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0097e28e66fd4ac5899095faa87f950b, effca16e25954b7b98ee12fc839a4534; no voters:
W20260924 16:11:27.708525 3931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:27.708595 3931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:27.708362 3932 sys_catalog.cc:455] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.710687 3932 sys_catalog.cc:458] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.707655 3692 master_runner.cc:186] Error getting master registration for 127.13.8.125:34157: OK, instance_id { permanent_uuid: "456825c477ba4fbebe5a079071af8322" instance_seqno: 1790266287430128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 16:11:27.711015 3692 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 16:11:27.711356 3944 catalog_manager.cc:1558] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:27.711422 3944 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:27.704758 3734 leader_election.cc:288] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [CANDIDATE]: Term 1 election: Requested vote from peers 456825c477ba4fbebe5a079071af8322 (127.13.8.125:34157), effca16e25954b7b98ee12fc839a4534 (127.13.8.124:39889)
I20260924 16:11:27.714250 3835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: Neither blocks nor log segments found. Creating new log.
I20260924 16:11:27.715282 3835 log.cc:824] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: Log is configured to *not* fsync() on all Append() calls
I20260924 16:11:27.714442 3841 master_runner.cc:186] Error getting master registration for 127.13.8.125:34157: OK, instance_id { permanent_uuid: "456825c477ba4fbebe5a079071af8322" instance_seqno: 1790266287430128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 16:11:27.717860 3841 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 16:11:27.718482 3835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: No bootstrap required, opened a new log
I20260924 16:11:27.723587 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.713766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50256 (local address 127.13.8.126:33925)
0924 16:11:27.713893 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.713898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.714248 (+ 350us) server_negotiation.cc:407] Connection header received
0924 16:11:27.714456 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.714463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.714515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.714566 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.715608 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.716382 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.717561 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.717885 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.717969 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.719876 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.719880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.719883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.719907 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.721628 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.722111 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.722114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.722116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.722198 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.722540 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.722543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.722545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.722806 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.722928 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.723261 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.723385 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 16:11:27.727046 3835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.727330 3835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456825c477ba4fbebe5a079071af8322, State: Initialized, Role: FOLLOWER
I20260924 16:11:27.727694 3835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.729235 3835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 16:11:27.729660 3809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0097e28e66fd4ac5899095faa87f950b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "456825c477ba4fbebe5a079071af8322"
I20260924 16:11:27.729938 3809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [term 0 FOLLOWER]: Advancing to term 1
I20260924 16:11:27.731606 3809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0097e28e66fd4ac5899095faa87f950b in term 1.
I20260924 16:11:27.732980 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.722989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52868 (local address 127.13.8.124:39889)
0924 16:11:27.723117 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.723122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.723697 (+ 575us) server_negotiation.cc:407] Connection header received
0924 16:11:27.723894 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.723897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.723945 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.724001 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.724881 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.725655 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.726895 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.727237 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.727316 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.729115 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.729119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.729121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.729151 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.731019 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.731534 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.731538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.731540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.731637 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.732019 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.732023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.732024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.732271 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.732389 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.732703 (+ 314us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.732823 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 16:11:27.734570 3955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.734701 3955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.735289 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.735456 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.735569 3945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 16:11:27.735661 3945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [term 1 LEADER]: Becoming Leader. State: Replica: 0097e28e66fd4ac5899095faa87f950b, State: Running, Role: LEADER
I20260924 16:11:27.736060 3945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } }
I20260924 16:11:27.738533 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0097e28e66fd4ac5899095faa87f950b. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.738674 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:27.740599 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:27.715972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.13.8.125:34157)
0924 16:11:27.716081 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.716085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.722506 (+ 6421us) server_negotiation.cc:407] Connection header received
0924 16:11:27.722719 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.722722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.722773 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.722855 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.724347 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.725121 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.726399 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:27.726770 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:27.726884 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:27.736750 (+ 9866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 16:11:27.736754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 16:11:27.736756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 16:11:27.736787 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 16:11:27.738476 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.739102 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.739105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.739107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.739193 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 16:11:27.739501 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 16:11:27.739504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 16:11:27.739506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 16:11:27.739771 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 16:11:27.739897 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:27.740268 (+ 371us) server_negotiation.cc:299] Negotiation successful
0924 16:11:27.740404 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 16:11:27.751046 3960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 16:11:27.751928 3960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260924 16:11:27.759177 3969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 16:11:27.759359 3969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 16:11:27.766556 3809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [term 1 FOLLOWER]: Refusing update from remote peer 0097e28e66fd4ac5899095faa87f950b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:11:27.766757 3886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [term 1 FOLLOWER]: Refusing update from remote peer 0097e28e66fd4ac5899095faa87f950b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 16:11:27.767292 3920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [LEADER]: Connected to new peer: Peer: permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:11:27.771580 3955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0097e28e66fd4ac5899095faa87f950b. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.771719 3955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.771838 3920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [LEADER]: Connected to new peer: Peer: permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 16:11:27.777617 3932 sys_catalog.cc:455] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0097e28e66fd4ac5899095faa87f950b. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.777753 3932 sys_catalog.cc:458] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.783347 3955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.783490 3955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.783917 3920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.784058 3920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:27.784168 3970 mvcc.cc:205] Tried to move back new op lower bound from 7332930714681184256 to 7332930714572181504. Current Snapshot: MvccSnapshot[applied={T|T < 7332930714681184256}]
I20260924 16:11:27.785995 3972 mvcc.cc:205] Tried to move back new op lower bound from 7332930714681184256 to 7332930714572181504. Current Snapshot: MvccSnapshot[applied={T|T < 7332930714681184256}]
I20260924 16:11:27.790966 3920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.791112 3920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b [sys.catalog]: This master's current role is: LEADER
I20260924 16:11:27.792259 3932 sys_catalog.cc:455] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0097e28e66fd4ac5899095faa87f950b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0097e28e66fd4ac5899095faa87f950b" member_type: VOTER last_known_addr { host: "127.13.8.126" port: 33925 } } peers { permanent_uuid: "456825c477ba4fbebe5a079071af8322" member_type: VOTER last_known_addr { host: "127.13.8.125" port: 34157 } } peers { permanent_uuid: "effca16e25954b7b98ee12fc839a4534" member_type: VOTER last_known_addr { host: "127.13.8.124" port: 39889 } } }
I20260924 16:11:27.792378 3932 sys_catalog.cc:458] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 16:11:27.794698 3960 catalog_manager.cc:1347] Generated new cluster ID: 3f8c999ed9554ca7872bce8cec3788ee
I20260924 16:11:27.794772 3960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 16:11:27.808583 3960 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 16:11:27.809887 3960 catalog_manager.cc:1504] Loading token signing keys...
I20260924 16:11:27.832789 3960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0097e28e66fd4ac5899095faa87f950b: Generated new TSK 0
I20260924 16:11:27.833806 3960 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 16:11:27.868331 3763 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.865812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.13.8.126:33925)
0924 16:11:27.865942 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.865947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.867249 (+ 1302us) server_negotiation.cc:407] Connection header received
0924 16:11:27.867443 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.867446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.867496 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.867549 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.868188 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50266: BlockingRecv error: recv got EOF from 127.0.0.1:50266 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260924 16:11:27.869729 3917 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.866682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.13.8.124:39889)
0924 16:11:27.866790 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.866795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.868819 (+ 2024us) server_negotiation.cc:407] Connection header received
0924 16:11:27.868983 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.868986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.869033 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.869081 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.869611 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52882: BlockingRecv error: recv got EOF from 127.0.0.1:52882 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260924 16:11:27.871693 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.870396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.13.8.125:34157)
0924 16:11:27.870504 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.870509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.870801 (+ 292us) server_negotiation.cc:407] Connection header received
0924 16:11:27.870973 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.870977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.871022 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.871069 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.871530 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51258: BlockingRecv error: recv got EOF from 127.0.0.1:51258 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260924 16:11:27.875995 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.874529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.13.8.125:34157)
0924 16:11:27.874640 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.874644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.875044 (+ 400us) server_negotiation.cc:407] Connection header received
0924 16:11:27.875265 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.875268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.875354 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.875411 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.875902 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51268: BlockingRecv error: recv got EOF from 127.0.0.1:51268 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 16:11:27.881001 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.879499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.13.8.125:34157)
0924 16:11:27.879609 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.879613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.880075 (+ 462us) server_negotiation.cc:407] Connection header received
0924 16:11:27.880285 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.880289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.880343 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.880395 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.880898 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51280: BlockingRecv error: recv got EOF from 127.0.0.1:51280 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260924 16:11:27.886868 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.885325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.13.8.125:34157)
0924 16:11:27.885437 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.885441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.885929 (+ 488us) server_negotiation.cc:407] Connection header received
0924 16:11:27.886123 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.886127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.886176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.886253 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.886764 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51294: BlockingRecv error: recv got EOF from 127.0.0.1:51294 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260924 16:11:27.894971 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.893176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.13.8.125:34157)
0924 16:11:27.893309 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.893315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.893807 (+ 492us) server_negotiation.cc:407] Connection header received
0924 16:11:27.893994 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.893999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.894063 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.894120 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.894861 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51306: BlockingRecv error: recv got EOF from 127.0.0.1:51306 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260924 16:11:27.907878 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.905758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.13.8.125:34157)
0924 16:11:27.905913 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.905918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.906653 (+ 735us) server_negotiation.cc:407] Connection header received
0924 16:11:27.906907 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.906911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.906969 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.907027 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.907742 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51308: BlockingRecv error: recv got EOF from 127.0.0.1:51308 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 16:11:27.928781 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.926932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.13.8.125:34157)
0924 16:11:27.927057 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.927062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.927660 (+ 598us) server_negotiation.cc:407] Connection header received
0924 16:11:27.927912 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.927915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.927983 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.928038 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.928652 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51322: BlockingRecv error: recv got EOF from 127.0.0.1:51322 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260924 16:11:27.964905 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:27.963636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.13.8.125:34157)
0924 16:11:27.963781 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:27.963785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:27.963962 (+ 177us) server_negotiation.cc:407] Connection header received
0924 16:11:27.964104 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:27.964107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:27.964168 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:27.964265 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:27.964780 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51336: BlockingRecv error: recv got EOF from 127.0.0.1:51336 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260924 16:11:28.032462 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:28.031252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.13.8.125:34157)
0924 16:11:28.031437 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.031443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.031573 (+ 130us) server_negotiation.cc:407] Connection header received
0924 16:11:28.031702 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.031705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.031764 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.031867 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:28.032336 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51338: BlockingRecv error: recv got EOF from 127.0.0.1:51338 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260924 16:11:28.164057 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:28.162776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.13.8.125:34157)
0924 16:11:28.162969 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.162974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.163112 (+ 138us) server_negotiation.cc:407] Connection header received
0924 16:11:28.163304 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.163307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.163363 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.163452 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:28.163935 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51348: BlockingRecv error: recv got EOF from 127.0.0.1:51348 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260924 16:11:28.423231 3840 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 16:11:28.422145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.13.8.125:34157)
0924 16:11:28.422354 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.422358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.422375 (+ 17us) server_negotiation.cc:407] Connection header received
0924 16:11:28.422513 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.422516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.422573 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.422672 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 16:11:28.423088 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51350: BlockingRecv error: recv got EOF from 127.0.0.1:51350 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
I20260924 16:11:28.712774 3944 catalog_manager.cc:1259] Loaded cluster ID: 3f8c999ed9554ca7872bce8cec3788ee
I20260924 16:11:28.712858 3944 catalog_manager.cc:1552] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: loading cluster ID for follower catalog manager: success
I20260924 16:11:28.714540 3944 catalog_manager.cc:1574] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: acquiring CA information for follower catalog manager: success
I20260924 16:11:28.716051 3944 catalog_manager.cc:1602] T 00000000000000000000000000000000 P effca16e25954b7b98ee12fc839a4534: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:11:28.760864 3969 catalog_manager.cc:1259] Loaded cluster ID: 3f8c999ed9554ca7872bce8cec3788ee
I20260924 16:11:28.760949 3969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: loading cluster ID for follower catalog manager: success
I20260924 16:11:28.762813 3969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: acquiring CA information for follower catalog manager: success
I20260924 16:11:28.764002 3969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 456825c477ba4fbebe5a079071af8322: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 16:11:28.941675 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:28.937247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.13.8.125:34157)
0924 16:11:28.937525 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.937529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.937545 (+ 16us) server_negotiation.cc:407] Connection header received
0924 16:11:28.937692 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.937695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.937832 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.937934 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:28.938168 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.938915 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.940041 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.940302 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.940437 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:28.940750 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:28.941214 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:28.941317 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:28.941395 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 16:11:28.941449 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260924 16:11:28.948525 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:28.944885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52888 (local address 127.13.8.124:39889)
0924 16:11:28.945116 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.945120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.945132 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:11:28.945220 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.945222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.945344 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.945441 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:28.945637 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.946140 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.947046 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.947274 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.947351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:28.947653 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:28.948089 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:28.948194 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:28.948259 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 16:11:28.948309 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260924 16:11:28.955572 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:28.951986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.13.8.124:39889)
0924 16:11:28.952087 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.952096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.952108 (+ 12us) server_negotiation.cc:407] Connection header received
0924 16:11:28.952151 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.952153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.952191 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.952252 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:28.952521 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.953221 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.954442 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.954659 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.954777 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:28.955057 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:28.955147 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:28.955217 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:28.955373 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 16:11:28.955429 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 16:11:28.956701 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:28.951784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.13.8.125:34157)
0924 16:11:28.951934 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.951939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.952747 (+ 808us) server_negotiation.cc:407] Connection header received
0924 16:11:28.952887 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.952890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.952961 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.953031 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:28.953394 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.954257 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.955319 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.955635 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.955741 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:28.956092 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:28.956167 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:28.956245 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:28.956532 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 16:11:28.956592 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 16:11:28.956787 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 16:11:28.951548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.13.8.126:33925)
0924 16:11:28.952270 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0924 16:11:28.952275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 16:11:28.952502 (+ 227us) server_negotiation.cc:407] Connection header received
0924 16:11:28.952647 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 16:11:28.952650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 16:11:28.952814 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 16:11:28.952941 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 16:11:28.953195 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.953810 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.954943 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 16:11:28.955235 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 16:11:28.955346 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 16:11:28.955810 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 16:11:28.956185 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 16:11:28.956282 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 16:11:28.956584 (+ 302us) server_negotiation.cc:299] Negotiation successful
0924 16:11:28.956640 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":82,"threads_started":1}
I20260924 16:11:28.960315 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3692
I20260924 16:11:28.968725 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3767
I20260924 16:11:28.975881 13345 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf9hPi5/build/debug/bin/kudu with pid 3841
2026-09-24T16:11:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3115 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6383 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287628 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST